[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:38 dist-test-slave-x2t8 krb5kdc[16216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:38 dist-test-slave-x2t8 krb5kdc[16216](info): set up 2 sockets
Sep 28 23:06:38 dist-test-slave-x2t8 krb5kdc[16216](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 23:06:39.792788 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:39 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:39.919364 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37097
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:40.002130 16229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:40.002324 16229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:40.002346 16229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:40.003417 16229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:40.003453 16229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:40.003468 16229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:40.003479 16229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:40.003491 16229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:40.004966 16229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:40.005158 16229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:40.005326 16229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:40.010625 16234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:40.010653 16235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.010686 16229 server_base.cc:1034] running on GCE node
W20260928 23:06:40.010655 16237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.011066 16229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:40.011451 16229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:40.012614 16229 hybrid_clock.cc:648] HybridClock initialized: now 1790636800012598 us; error 36 us; skew 500 ppm
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:40.015267 16229 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:40.015636 16229 webserver.cc:460] Webserver started at http://127.15.212.126:33585/ using document root <none> and password file <none>
I20260928 23:06:40.015828 16229 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:40.015887 16229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:40.015974 16229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:40.017007 16229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "ce92b38fa93a480d8b0fc9475430f351"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "3b0a3d3b8039a0a0d472df75fe197cf3"
server_key_iv: "dceff6a8ab186ca1c85f0fa58668b3a9"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.017306 16229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "ce92b38fa93a480d8b0fc9475430f351"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "3b0a3d3b8039a0a0d472df75fe197cf3"
server_key_iv: "dceff6a8ab186ca1c85f0fa58668b3a9"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.018543 16229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:40.019217 16244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.019395 16229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:40.019465 16229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "ce92b38fa93a480d8b0fc9475430f351"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "3b0a3d3b8039a0a0d472df75fe197cf3"
server_key_iv: "dceff6a8ab186ca1c85f0fa58668b3a9"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.019555 16229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:40.040727 16229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:40.042449 16229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:40.042603 16229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:40.046383 16229 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:37097
I20260928 23:06:40.046452 16296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:37097 every 8 connection(s)
I20260928 23:06:40.046790 16229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:40.047650 16297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:40.049929 16297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351: Bootstrap starting.
I20260928 23:06:40.050487 16297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:40.050771 16297 log.cc:824] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:40.051443 16297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351: No bootstrap required, opened a new log
I20260928 23:06:40.053045 16297 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } }
I20260928 23:06:40.053205 16297 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:40.053253 16297 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce92b38fa93a480d8b0fc9475430f351, State: Initialized, Role: FOLLOWER
I20260928 23:06:40.053435 16297 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } }
I20260928 23:06:40.053534 16297 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:40.053576 16297 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:40.053629 16297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:40.053979 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16229
I20260928 23:06:40.054057 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:40.054234 16297 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } }
I20260928 23:06:40.054349 16297 leader_election.cc:302] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce92b38fa93a480d8b0fc9475430f351; no voters:
I20260928 23:06:40.054508 16297 leader_election.cc:288] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:40.054546 16300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:40.054667 16300 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [term 1 LEADER]: Becoming Leader. State: Replica: ce92b38fa93a480d8b0fc9475430f351, State: Running, Role: LEADER
I20260928 23:06:40.054773 16297 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:40.054746 16300 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } }
I20260928 23:06:40.054808 16209 external_mini_cluster.cc:1428] Setting key 11201711aa138a8afe58f55fd43356d9
I20260928 23:06:40.055088 16300 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce92b38fa93a480d8b0fc9475430f351. Latest consensus state: current_term: 1 leader_uuid: "ce92b38fa93a480d8b0fc9475430f351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } } }
I20260928 23:06:40.055168 16300 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:40.055330 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce92b38fa93a480d8b0fc9475430f351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce92b38fa93a480d8b0fc9475430f351" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37097 } } }
I20260928 23:06:40.055459 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:40.055371 16305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:40.056315 16305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:40.057950 16305 catalog_manager.cc:1347] Generated new cluster ID: 000cfdb6947940cfba62f01b09a1951b
I20260928 23:06:40.058004 16305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:40.065198 16305 catalog_manager.cc:1370] Generated new certificate authority record
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:40.065852 16305 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:40.069546 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.056315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32956 (local address 127.15.212.126:37097)
0928 23:06:40.056570 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.056579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.056614 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:06:40.058606 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.058612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.058655 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.058735 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.060190 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.061147 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.062143 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.062534 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.062673 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.066016 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.066019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.066019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.066028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.067681 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.068113 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.068116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.068117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.068208 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.068483 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.068484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.068484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.068687 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.068866 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.069119 (+ 253us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.069289 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":67,"threads_started":1}
I20260928 23:06:40.075845 16305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce92b38fa93a480d8b0fc9475430f351: Generated new TSK 0
I20260928 23:06:40.076000 16305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:40.107056 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:40.190915 16323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:40.191100 16323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:40.191123 16323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:40.192271 16323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:40.192313 16323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:40.192340 16323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:40.193823 16323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:40.194068 16323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:40.194298 16323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:40.199237 16331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:40.199256 16328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.199463 16323 server_base.cc:1034] running on GCE node
W20260928 23:06:40.199579 16329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.199805 16323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:40.200060 16323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:40.201222 16323 hybrid_clock.cc:648] HybridClock initialized: now 1790636800201223 us; error 41 us; skew 500 ppm
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:40.203593 16323 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:40.204054 16323 webserver.cc:460] Webserver started at http://127.15.212.65:41939/ using document root <none> and password file <none>
I20260928 23:06:40.204286 16323 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:40.204355 16323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:40.204527 16323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:40.205526 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "f84f10a2f70c456fb4da64847781f5e0"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "8fed270f1b253fad345f2bc8c204d9ff"
server_key_iv: "a656ba5425aa61d17493f75f9b897834"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.205886 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "f84f10a2f70c456fb4da64847781f5e0"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "8fed270f1b253fad345f2bc8c204d9ff"
server_key_iv: "a656ba5425aa61d17493f75f9b897834"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.207178 16323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:06:40.207844 16338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.208009 16323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:40.208101 16323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "f84f10a2f70c456fb4da64847781f5e0"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "8fed270f1b253fad345f2bc8c204d9ff"
server_key_iv: "a656ba5425aa61d17493f75f9b897834"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.208186 16323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:40.223029 16323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:40.225343 16323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:40.225450 16323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:40.225649 16323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:40.225957 16323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:40.225991 16323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.226014 16323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:40.226027 16323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.231432 16323 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:39393
I20260928 23:06:40.231505 16451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:39393 every 8 connection(s)
I20260928 23:06:40.231818 16323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:40.240373 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.232493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:55263 (local address 127.15.212.126:37097)
0928 23:06:40.232542 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.232544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.232814 (+ 270us) server_negotiation.cc:407] Connection header received
0928 23:06:40.233331 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.233333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.233346 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.233393 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.234312 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.234804 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.235786 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.235989 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.236072 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.238134 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.238135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.238136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.238147 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.239297 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.239641 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.239642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.239642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.239687 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.239885 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.239886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.239887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.239978 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.240026 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.240251 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.240312 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:06:40.240564 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.232521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37097 (local address 127.15.212.65:55263)
0928 23:06:40.232695 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:40.232729 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:40.233220 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:40.233471 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:40.233482 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:40.234027 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:40.234208 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.234213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.234909 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.234910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.235693 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.235695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.236081 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.236082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.236208 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.236694 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:40.236702 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:40.238040 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:40.239368 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.239373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.239375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.239579 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.239742 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.239742 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.239743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.239833 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.240030 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:40.240033 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:40.240137 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 23:06:40.240224 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 23:06:40.240334 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260928 23:06:40.241030 16452 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37097
I20260928 23:06:40.241130 16452 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:40.241312 16452 heartbeater.cc:511] Master 127.15.212.126:37097 requested a full tablet report, sending...
I20260928 23:06:40.241498 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16323
I20260928 23:06:40.241590 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:40.241756 16209 external_mini_cluster.cc:1428] Setting key a5c70d25310f15871e7501e2e82ef3d5
I20260928 23:06:40.241847 16261 ts_manager.cc:194] Registered new tserver with Master: f84f10a2f70c456fb4da64847781f5e0 (127.15.212.65:39393)
I20260928 23:06:40.242654 16261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:55263
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:40.265426 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:40.345117 16457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:40.345323 16457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:40.345346 16457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:40.346422 16457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:40.346457 16457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:40.346482 16457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:40.347831 16457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:40.348016 16457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:40.348186 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:40.353312 16462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:40.353322 16463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.353338 16457 server_base.cc:1034] running on GCE node
W20260928 23:06:40.353322 16465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.353652 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:40.353857 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:40.355019 16457 hybrid_clock.cc:648] HybridClock initialized: now 1790636800355001 us; error 31 us; skew 500 ppm
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:40.357548 16457 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:40.358023 16457 webserver.cc:460] Webserver started at http://127.15.212.66:36673/ using document root <none> and password file <none>
I20260928 23:06:40.358263 16457 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:40.358306 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:40.358456 16457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:40.359483 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "3d16ce0166e544e8a105c90c33ec4d28"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "820c50b65b2c03cfb50a7ef4ec527719"
server_key_iv: "74db1cfae91ed15ba65cad0953123b79"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.359843 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "3d16ce0166e544e8a105c90c33ec4d28"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "820c50b65b2c03cfb50a7ef4ec527719"
server_key_iv: "74db1cfae91ed15ba65cad0953123b79"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.361220 16457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:40.361927 16472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.362114 16457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:40.362198 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "3d16ce0166e544e8a105c90c33ec4d28"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "820c50b65b2c03cfb50a7ef4ec527719"
server_key_iv: "74db1cfae91ed15ba65cad0953123b79"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.362250 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:40.376693 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:40.379093 16457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:40.379216 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:40.379403 16457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:40.379726 16457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:40.379763 16457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.379808 16457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:40.379835 16457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.385551 16457 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:39441
I20260928 23:06:40.385648 16585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:39441 every 8 connection(s)
I20260928 23:06:40.385950 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:40.389736 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16457
I20260928 23:06:40.389837 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:40.389998 16209 external_mini_cluster.cc:1428] Setting key a8267a9c710629e59f2054dec6785d33
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:40.395382 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.386688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:52135 (local address 127.15.212.126:37097)
0928 23:06:40.386732 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.386734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.386962 (+ 228us) server_negotiation.cc:407] Connection header received
0928 23:06:40.387487 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.387488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.387500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.387546 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.388541 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.389032 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.390094 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.390330 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.390401 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.392532 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.392533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.392534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.392541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.393935 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.394390 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.394391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.394391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.394453 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.394740 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.394741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.394741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.394860 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.394936 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.395213 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.395289 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:06:40.395592 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.386652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37097 (local address 127.15.212.66:52135)
0928 23:06:40.386845 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:40.386879 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:40.387383 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:40.387616 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:40.387621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:40.388211 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:40.388400 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.388404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.389140 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.389141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.389996 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.389999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.390408 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.390409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.390573 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.391096 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:40.391105 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:40.392394 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:40.394061 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.394068 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.394070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.394308 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.394548 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.394549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.394549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.394661 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.394927 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:40.394933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:40.395077 (+ 144us) client_negotiation.cc:769] Sending connection context
0928 23:06:40.395191 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 23:06:40.395346 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260928 23:06:40.396109 16586 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37097
I20260928 23:06:40.396214 16586 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:40.396405 16586 heartbeater.cc:511] Master 127.15.212.126:37097 requested a full tablet report, sending...
I20260928 23:06:40.396854 16261 ts_manager.cc:194] Registered new tserver with Master: 3d16ce0166e544e8a105c90c33ec4d28 (127.15.212.66:39441)
I20260928 23:06:40.397378 16261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:52135
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:40.415032 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:40.496637 16591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:40.496837 16591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:40.496861 16591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:40.498031 16591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:40.498076 16591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:40.498136 16591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:40.499634 16591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37097
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:40.499857 16591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:40.500102 16591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:40.505264 16596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.505359 16591 server_base.cc:1034] running on GCE node
W20260928 23:06:40.505306 16597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:40.505455 16599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:40.505704 16591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:40.505946 16591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:40.507119 16591 hybrid_clock.cc:648] HybridClock initialized: now 1790636800507108 us; error 24 us; skew 500 ppm
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:40.509610 16591 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:40.510085 16591 webserver.cc:460] Webserver started at http://127.15.212.67:40603/ using document root <none> and password file <none>
I20260928 23:06:40.510313 16591 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:40.510378 16591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:40.510526 16591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:40.511595 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "2e505e842f894c60aabb3bdce5a227c3"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "2e2a4bc9dba5d4b81fe841331324d4a6"
server_key_iv: "cff0c34e4db3c26cbe42e30fccea4f0a"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.511958 16591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "2e505e842f894c60aabb3bdce5a227c3"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "2e2a4bc9dba5d4b81fe841331324d4a6"
server_key_iv: "cff0c34e4db3c26cbe42e30fccea4f0a"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.513470 16591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:40.514187 16606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.514353 16591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.514456 16591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "2e505e842f894c60aabb3bdce5a227c3"
format_stamp: "Formatted at 2026-09-28 23:06:40 on dist-test-slave-x2t8"
server_key: "2e2a4bc9dba5d4b81fe841331324d4a6"
server_key_iv: "cff0c34e4db3c26cbe42e30fccea4f0a"
server_key_version: "encryptionkey@0"
I20260928 23:06:40.514528 16591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:40.528168 16591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:40.530049 16591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:40.530205 16591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:40.530439 16591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:40.530745 16591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:40.530797 16591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.530838 16591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:40.530864 16591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:40.536799 16591 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:46483
I20260928 23:06:40.536842 16719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:46483 every 8 connection(s)
I20260928 23:06:40.537150 16591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:40.539768 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16591
I20260928 23:06:40.539862 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:40.540014 16209 external_mini_cluster.cc:1428] Setting key 040061e3f18ffe9235c26b19390efe8c
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:40.546594 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.537871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:38543 (local address 127.15.212.126:37097)
0928 23:06:40.537932 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.537934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.538129 (+ 195us) server_negotiation.cc:407] Connection header received
0928 23:06:40.538568 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.538569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.538580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.538636 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.539489 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.540265 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.541589 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.541773 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.541828 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.543840 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.543841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.543842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.543849 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.545270 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.545680 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.545681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.545681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.545739 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.545985 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.545986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.545986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.546094 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.546165 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.546422 (+ 257us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.546497 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:06:40.546782 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.537841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37097 (local address 127.15.212.67:38543)
0928 23:06:40.537985 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:40.538024 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:40.538481 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:40.538687 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:40.538690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:40.539232 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:40.539416 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.539420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.540412 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.540413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.541517 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:40.541520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.541844 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.541845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:40.541935 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.542445 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:40.542457 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:40.543747 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:40.545366 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.545372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.545375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.545588 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.545802 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:40.545803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:40.545804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:40.545911 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:40.546157 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:40.546162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:40.546296 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 23:06:40.546404 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 23:06:40.546554 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260928 23:06:40.547183 16720 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37097
I20260928 23:06:40.547269 16720 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:40.547405 16720 heartbeater.cc:511] Master 127.15.212.126:37097 requested a full tablet report, sending...
I20260928 23:06:40.547727 16261 ts_manager.cc:194] Registered new tserver with Master: 2e505e842f894c60aabb3bdce5a227c3 (127.15.212.67:46483)
I20260928 23:06:40.548041 16261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:38543
I20260928 23:06:40.552081 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:40.574506 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.567274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.15.212.65:39393)
0928 23:06:40.567427 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.567432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.567451 (+ 19us) server_negotiation.cc:407] Connection header received
0928 23:06:40.567478 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.567480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.567493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.567563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.568086 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.568868 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.569575 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.570001 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.570060 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.572144 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.572152 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.572152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.572158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.573582 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.573877 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.573880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.573880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.573930 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.574085 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.574086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.574086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.574211 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.574310 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.574371 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.574429 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
W20260928 23:06:40.574883 16366 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49818
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:40.597070 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:40.589986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.15.212.65:39393)
0928 23:06:40.590048 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:40.590050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:40.590062 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:06:40.590091 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:40.590092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:40.590104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:40.590194 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:40.590840 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.591330 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.592073 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:40.592289 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:40.592384 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:40.594535 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:40.594537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:40.594538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:40.594546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:40.595866 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.596246 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.596247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.596248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.596306 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:40.596556 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:40.596557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:40.596558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:40.596667 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:40.596790 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:40.596880 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 23:06:40.596964 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:06:40.597666 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16323
I20260928 23:06:40.602535 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16457
I20260928 23:06:40.606846 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16591
I20260928 23:06:40.611069 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16229
2026-09-28T23:06:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1857 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16734](info): set up 2 sockets
Sep 28 23:06:40 dist-test-slave-x2t8 krb5kdc[16734](info): commencing operation
krb5kdc: starting...
W20260928 23:06:41.647909 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:41 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:41.762950 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44337
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:44337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:41.841881 16747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:41.842084 16747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:41.842106 16747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:41.843173 16747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:41.843205 16747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:41.843220 16747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:41.843232 16747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:41.843245 16747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:41.844691 16747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:44337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:41.844877 16747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:41.845033 16747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:41.849557 16752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:41.849607 16753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:41.849586 16755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:41.849706 16747 server_base.cc:1034] running on GCE node
I20260928 23:06:41.850047 16747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:41.850338 16747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:41.851477 16747 hybrid_clock.cc:648] HybridClock initialized: now 1790636801851467 us; error 30 us; skew 500 ppm
Sep 28 23:06:41 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636801, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:41.853842 16747 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:41.854274 16747 webserver.cc:460] Webserver started at http://127.15.212.126:44219/ using document root <none> and password file <none>
I20260928 23:06:41.854461 16747 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:41.854501 16747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:41.854584 16747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:41.855535 16747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "931028dad42a4f3184499c4a1eb31968"
format_stamp: "Formatted at 2026-09-28 23:06:41 on dist-test-slave-x2t8"
server_key: "04af6cbd4f2fd8afbdad11f26d42281d"
server_key_iv: "abb1746a0a0946c9fcd2b152d12377f3"
server_key_version: "encryptionkey@0"
I20260928 23:06:41.855836 16747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "931028dad42a4f3184499c4a1eb31968"
format_stamp: "Formatted at 2026-09-28 23:06:41 on dist-test-slave-x2t8"
server_key: "04af6cbd4f2fd8afbdad11f26d42281d"
server_key_iv: "abb1746a0a0946c9fcd2b152d12377f3"
server_key_version: "encryptionkey@0"
I20260928 23:06:41.857115 16747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:06:41.857846 16762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:41.858011 16747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:41.858117 16747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "931028dad42a4f3184499c4a1eb31968"
format_stamp: "Formatted at 2026-09-28 23:06:41 on dist-test-slave-x2t8"
server_key: "04af6cbd4f2fd8afbdad11f26d42281d"
server_key_iv: "abb1746a0a0946c9fcd2b152d12377f3"
server_key_version: "encryptionkey@0"
I20260928 23:06:41.858194 16747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:41.895799 16747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:41.898041 16747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:41.898197 16747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:41.902129 16747 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:44337
I20260928 23:06:41.902172 16814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:44337 every 8 connection(s)
I20260928 23:06:41.902565 16747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:41.903407 16815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:41.905776 16815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968: Bootstrap starting.
I20260928 23:06:41.906358 16815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:41.906648 16815 log.cc:824] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:41.907336 16815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968: No bootstrap required, opened a new log
I20260928 23:06:41.908012 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16747
I20260928 23:06:41.908078 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:41.908247 16209 external_mini_cluster.cc:1428] Setting key 2e8546976505f28597873bd847680237
I20260928 23:06:41.909472 16815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } }
I20260928 23:06:41.909651 16815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:41.909708 16815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931028dad42a4f3184499c4a1eb31968, State: Initialized, Role: FOLLOWER
I20260928 23:06:41.909852 16815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } }
I20260928 23:06:41.909930 16815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:41.909952 16815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:41.910012 16815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:41.911100 16815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } }
I20260928 23:06:41.911211 16815 leader_election.cc:302] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 931028dad42a4f3184499c4a1eb31968; no voters:
I20260928 23:06:41.911365 16815 leader_election.cc:288] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:41.911419 16820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:41.911550 16820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [term 1 LEADER]: Becoming Leader. State: Replica: 931028dad42a4f3184499c4a1eb31968, State: Running, Role: LEADER
I20260928 23:06:41.911592 16815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:41.911638 16820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } }
I20260928 23:06:41.911972 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "931028dad42a4f3184499c4a1eb31968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } } }
I20260928 23:06:41.912046 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:41.912238 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931028dad42a4f3184499c4a1eb31968. Latest consensus state: current_term: 1 leader_uuid: "931028dad42a4f3184499c4a1eb31968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "931028dad42a4f3184499c4a1eb31968" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44337 } } }
I20260928 23:06:41.912310 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:41.912427 16827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:41.912946 16827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:41.914781 16827 catalog_manager.cc:1347] Generated new cluster ID: 1b0490f944bb45ca8f9325dd1e091ea6
I20260928 23:06:41.914834 16827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:06:41 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:41.922706 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:41.908795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.15.212.126:44337)
0928 23:06:41.908978 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:41.908988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:41.909025 (+ 37us) server_negotiation.cc:407] Connection header received
0928 23:06:41.909428 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:41.909437 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:41.909501 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:41.909655 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:41.910372 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:41.911843 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:41.913008 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:41.913631 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:41.913766 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:41.919075 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:41.919080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:41.919080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:41.919091 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:41.921025 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:41.921437 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:41.921440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:41.921441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:41.921559 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:41.921785 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:41.921785 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:41.921786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:41.922069 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:41.922203 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:41.922341 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 23:06:41.922483 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260928 23:06:41.924798 16827 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:41.925469 16827 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:41.931710 16827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 931028dad42a4f3184499c4a1eb31968: Generated new TSK 0
I20260928 23:06:41.931878 16827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:41.961326 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:42.042388 16841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:42.042645 16841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:42.042688 16841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:42.043938 16841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:42.044008 16841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:42.044056 16841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:42.045764 16841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:42.046042 16841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:42.046277 16841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:42.051560 16847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:42.051726 16841 server_base.cc:1034] running on GCE node
W20260928 23:06:42.051565 16846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:42.051602 16849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:42.052048 16841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:42.052256 16841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:42.053587 16841 hybrid_clock.cc:648] HybridClock initialized: now 1790636802053575 us; error 48 us; skew 500 ppm
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:42.056211 16841 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:42.056706 16841 webserver.cc:460] Webserver started at http://127.15.212.65:33591/ using document root <none> and password file <none>
I20260928 23:06:42.056948 16841 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:42.056993 16841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:42.057181 16841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:42.058261 16841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "5aa8a5b05edb446886c76dc3222fc53e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "b0be709fb9233a2b4d52cf0af912d85e"
server_key_iv: "aba4ccbab04ca24071e29df7110230af"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.058617 16841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "5aa8a5b05edb446886c76dc3222fc53e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "b0be709fb9233a2b4d52cf0af912d85e"
server_key_iv: "aba4ccbab04ca24071e29df7110230af"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.059944 16841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:42.060784 16856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.060976 16841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:42.061040 16841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "5aa8a5b05edb446886c76dc3222fc53e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "b0be709fb9233a2b4d52cf0af912d85e"
server_key_iv: "aba4ccbab04ca24071e29df7110230af"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.061127 16841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:42.078231 16841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:42.080066 16841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:42.080233 16841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:42.080554 16841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:42.080880 16841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:42.080914 16841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.080936 16841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:42.080950 16841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.086670 16841 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:37641
I20260928 23:06:42.086766 16969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:37641 every 8 connection(s)
I20260928 23:06:42.087071 16841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:42.096522 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16841
I20260928 23:06:42.096556 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.087848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:35049 (local address 127.15.212.126:44337)
0928 23:06:42.087915 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.087917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.088128 (+ 211us) server_negotiation.cc:407] Connection header received
0928 23:06:42.089075 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.089076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.089089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.089138 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.089723 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.090214 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.091195 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.091387 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.091439 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.093785 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.093787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.093787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.093794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.095285 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.095649 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.095650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.095651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.095700 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.095905 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.095906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.095906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.096007 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.096056 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.096294 (+ 238us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.096419 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:06:42.096612 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:42.096665 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.087837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44337 (local address 127.15.212.65:35049)
0928 23:06:42.088005 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.088041 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.088982 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.089206 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.089209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.089454 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:42.089651 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.089656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.090293 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.090294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.091117 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.091120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.091459 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.091460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.091558 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.092119 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:42.092134 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:42.093687 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:42.095356 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.095361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.095363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.095569 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.095758 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.095759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.095759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.095853 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.096060 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:42.096064 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:42.096184 (+ 120us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.096265 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.096382 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260928 23:06:42.096784 16209 external_mini_cluster.cc:1428] Setting key 9a945ab5930910016778e520d338f274
I20260928 23:06:42.097223 16970 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44337
I20260928 23:06:42.097318 16970 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:42.097530 16970 heartbeater.cc:511] Master 127.15.212.126:44337 requested a full tablet report, sending...
I20260928 23:06:42.098124 16779 ts_manager.cc:194] Registered new tserver with Master: 5aa8a5b05edb446886c76dc3222fc53e (127.15.212.65:37641)
I20260928 23:06:42.099015 16779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:35049
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:42.123142 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:42.208396 16975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:42.208624 16975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:42.208647 16975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:42.209813 16975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:42.209857 16975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:42.209883 16975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:42.211282 16975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:42.211467 16975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:42.211630 16975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:42.216645 16983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:42.216612 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:42.216610 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:42.216846 16975 server_base.cc:1034] running on GCE node
I20260928 23:06:42.217056 16975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:42.217254 16975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:42.218353 16975 hybrid_clock.cc:648] HybridClock initialized: now 1790636802218342 us; error 36 us; skew 500 ppm
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:42.220942 16975 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:42.221432 16975 webserver.cc:460] Webserver started at http://127.15.212.66:39693/ using document root <none> and password file <none>
I20260928 23:06:42.221678 16975 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:42.221745 16975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:42.221922 16975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:42.223223 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "d991e952e74a4352aca893d908bc80b9"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "259399d7a516b6a1cf0c403acfe4a66f"
server_key_iv: "1e868aee335cee0a84a77e02cf378c2f"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.223587 16975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "d991e952e74a4352aca893d908bc80b9"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "259399d7a516b6a1cf0c403acfe4a66f"
server_key_iv: "1e868aee335cee0a84a77e02cf378c2f"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.225157 16975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260928 23:06:42.225845 16990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.225996 16975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:42.226092 16975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "d991e952e74a4352aca893d908bc80b9"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "259399d7a516b6a1cf0c403acfe4a66f"
server_key_iv: "1e868aee335cee0a84a77e02cf378c2f"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.226187 16975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:42.240146 16975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:42.242690 16975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:42.242861 16975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:42.243101 16975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:42.243434 16975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:42.243487 16975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.243530 16975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:42.243557 16975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.249655 16975 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:44395
I20260928 23:06:42.249707 17103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:44395 every 8 connection(s)
I20260928 23:06:42.250021 16975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:42.258922 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.250764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34947 (local address 127.15.212.126:44337)
0928 23:06:42.250830 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.250833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.251047 (+ 214us) server_negotiation.cc:407] Connection header received
0928 23:06:42.251936 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.251937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.251946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.252001 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.252661 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.253158 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.254114 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.254298 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.254350 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.256426 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.256428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.256428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.256484 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.257712 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.258086 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.258086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.258087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.258135 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.258342 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.258342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.258343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.258444 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.258492 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.258725 (+ 233us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.258788 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:06:42.259002 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 16975
I20260928 23:06:42.259009 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.250753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44337 (local address 127.15.212.66:34947)
0928 23:06:42.250919 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.250960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.251850 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.252067 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.252070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.252324 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:42.252584 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.252589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.253243 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.253244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.254040 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.254042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.254367 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.254367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.254463 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.254919 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:42.254927 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:42.256328 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:42.257785 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.257791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.257793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.258021 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.258193 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.258193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.258194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.258289 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.258497 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:42.258501 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:42.258611 (+ 110us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.258701 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.258817 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260928 23:06:42.259075 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:42.259225 16209 external_mini_cluster.cc:1428] Setting key 0fb9b3fd8f3c9c8be5266a10e5ce8c45
I20260928 23:06:42.259843 17104 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44337
I20260928 23:06:42.259948 17104 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:42.260133 17104 heartbeater.cc:511] Master 127.15.212.126:44337 requested a full tablet report, sending...
I20260928 23:06:42.260542 16779 ts_manager.cc:194] Registered new tserver with Master: d991e952e74a4352aca893d908bc80b9 (127.15.212.66:44395)
I20260928 23:06:42.260942 16779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:34947
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:42.283711 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:42.365895 17109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:42.366101 17109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:42.366123 17109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:42.367287 17109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:42.367331 17109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:42.367385 17109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:42.368904 17109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:42.369158 17109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:42.369411 17109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:42.374396 17115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:42.374406 17117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:42.374436 17109 server_base.cc:1034] running on GCE node
W20260928 23:06:42.374420 17114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:42.374785 17109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:42.375061 17109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:42.376225 17109 hybrid_clock.cc:648] HybridClock initialized: now 1790636802376226 us; error 37 us; skew 500 ppm
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:42.379227 17109 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:42.379652 17109 webserver.cc:460] Webserver started at http://127.15.212.67:39391/ using document root <none> and password file <none>
I20260928 23:06:42.379889 17109 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:42.379954 17109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:42.380091 17109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:42.381234 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "f1a1cf74389d4bf0a72732a963a0c15e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "c7e677dcc60bed2f4f2be307f3d5033a"
server_key_iv: "959e288c0a15ce848834f7b68bda88d4"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.381621 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "f1a1cf74389d4bf0a72732a963a0c15e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "c7e677dcc60bed2f4f2be307f3d5033a"
server_key_iv: "959e288c0a15ce848834f7b68bda88d4"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.383098 17109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:06:42.383812 17124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.384016 17109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:42.384109 17109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "f1a1cf74389d4bf0a72732a963a0c15e"
format_stamp: "Formatted at 2026-09-28 23:06:42 on dist-test-slave-x2t8"
server_key: "c7e677dcc60bed2f4f2be307f3d5033a"
server_key_iv: "959e288c0a15ce848834f7b68bda88d4"
server_key_version: "encryptionkey@0"
I20260928 23:06:42.384197 17109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:42.393496 17109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:42.395787 17109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:42.395947 17109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:42.396188 17109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:42.396597 17109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:42.396660 17109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.396703 17109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:42.396729 17109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:42.402714 17109 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:46715
I20260928 23:06:42.402776 17237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:46715 every 8 connection(s)
I20260928 23:06:42.403115 17109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:42.408170 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17109
I20260928 23:06:42.408275 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:42.408469 16209 external_mini_cluster.cc:1428] Setting key edcc5df6ec21c7056501c92dd9ff2910
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:42.413151 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.403926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:49401 (local address 127.15.212.126:44337)
0928 23:06:42.404010 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.404013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.404116 (+ 103us) server_negotiation.cc:407] Connection header received
0928 23:06:42.405053 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.405054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.405064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.405117 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.405702 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.406193 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.407209 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.407414 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.407468 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.409989 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.409991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.409991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.409998 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.411541 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.411971 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.411972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.411973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.412024 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.412418 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.412423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.412424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.412590 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.412647 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.412958 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.413041 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 23:06:42.413348 17240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.403843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44337 (local address 127.15.212.67:49401)
0928 23:06:42.403990 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.404040 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.404961 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.405176 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.405179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.405428 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:42.405628 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.405633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.406291 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.406292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.407131 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.407133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.407486 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.407486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.407612 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.408091 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:42.408101 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:42.409888 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:42.411627 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.411632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.411634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.411918 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.412192 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:42.412193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:42.412193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:42.412352 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:42.412664 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:42.412668 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:42.412790 (+ 122us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.412924 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.413107 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260928 23:06:42.413817 17238 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44337
I20260928 23:06:42.413929 17238 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:42.414108 17238 heartbeater.cc:511] Master 127.15.212.126:44337 requested a full tablet report, sending...
I20260928 23:06:42.414507 16779 ts_manager.cc:194] Registered new tserver with Master: f1a1cf74389d4bf0a72732a963a0c15e (127.15.212.67:46715)
I20260928 23:06:42.414976 16779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:49401
I20260928 23:06:42.420387 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:06:42.428465 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.422001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.15.212.126:44337)
0928 23:06:42.422087 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.422089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.422189 (+ 100us) server_negotiation.cc:407] Connection header received
0928 23:06:42.422291 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.422292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.422303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.422359 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.423029 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.423522 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.424242 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.424505 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.424585 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.425830 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.425831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.425831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.425838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.427268 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.427629 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.427630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.427631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.427691 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.427894 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.427895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.427895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.428043 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.428157 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.428258 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.428339 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 23:06:42.430848 16779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:42.432541 16779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:42.438972 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:37641 (local address 127.0.0.1:42548)
0928 23:06:42.435369 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.435378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.435421 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.435674 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.435675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.435677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:42.435716 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.435718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436917 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.436918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.437736 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.437738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438639 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.438717 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.438732 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.438814 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.438886 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260928 23:06:42.438977 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.15.212.65:37641)
0928 23:06:42.435320 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.435325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.435469 (+ 144us) server_negotiation.cc:407] Connection header received
0928 23:06:42.435510 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.435513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.435528 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.435593 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:42.435791 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436818 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.437846 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438550 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438628 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.438699 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.438839 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.438883 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":78,"threads_started":1}
I20260928 23:06:42.439227 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:44395 (local address 127.0.0.1:34524)
0928 23:06:42.434964 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.435008 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.435082 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.435346 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.435349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.435352 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:42.435424 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.435426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436708 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.436708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.438159 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.438160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438967 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.439043 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.439047 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.439100 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.439142 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":50,"thread_start_us":59,"threads_started":1}
I20260928 23:06:42.439240 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:46715 (local address 127.0.0.1:56774)
0928 23:06:42.435113 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.435122 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.435188 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.435451 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.435452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.435454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:42.435497 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.435499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436596 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.436597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.437995 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.437998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438957 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.439029 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.439034 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.439091 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.439148 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":56,"spinlock_wait_cycles":768,"thread_start_us":90,"threads_started":1}
I20260928 23:06:42.439409 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.15.212.67:46715)
0928 23:06:42.435082 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.435090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.435183 (+ 93us) server_negotiation.cc:407] Connection header received
0928 23:06:42.435257 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.435258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.435271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.435366 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:42.435658 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436488 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438090 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438781 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438868 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.438922 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.439276 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.439318 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260928 23:06:42.439713 17252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.434797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.15.212.66:44395)
0928 23:06:42.434972 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.434978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.435174 (+ 196us) server_negotiation.cc:407] Connection header received
0928 23:06:42.435224 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.435226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.435244 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.435325 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:42.435489 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.436610 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438240 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.438823 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.438882 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.438934 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.439362 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.439410 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260928 23:06:42.440172 17172 tablet_service.cc:1467] Processing CreateTablet for tablet 2b80bbf456d64361b3cccead438a59f5 (DEFAULT_TABLE table=test-table [id=e50d88ec016e48fbbc91a09a479df304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:42.440315 17038 tablet_service.cc:1467] Processing CreateTablet for tablet 2b80bbf456d64361b3cccead438a59f5 (DEFAULT_TABLE table=test-table [id=e50d88ec016e48fbbc91a09a479df304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:42.440562 17172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b80bbf456d64361b3cccead438a59f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:42.440629 17038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b80bbf456d64361b3cccead438a59f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:42.440855 16904 tablet_service.cc:1467] Processing CreateTablet for tablet 2b80bbf456d64361b3cccead438a59f5 (DEFAULT_TABLE table=test-table [id=e50d88ec016e48fbbc91a09a479df304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:42.441154 16904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b80bbf456d64361b3cccead438a59f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:42.443195 17255 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: Bootstrap starting.
I20260928 23:06:42.443801 17255 tablet_bootstrap.cc:654] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:42.444058 17256 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Bootstrap starting.
I20260928 23:06:42.444300 17257 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Bootstrap starting.
I20260928 23:06:42.444424 17255 log.cc:824] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:42.444998 17257 tablet_bootstrap.cc:654] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:42.445088 17256 tablet_bootstrap.cc:654] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:42.445255 17255 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: No bootstrap required, opened a new log
I20260928 23:06:42.445338 17255 ts_tablet_manager.cc:1397] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:42.445541 17257 log.cc:824] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:42.445674 17256 log.cc:824] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:42.446353 17256 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: No bootstrap required, opened a new log
I20260928 23:06:42.446355 17257 tablet_bootstrap.cc:492] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: No bootstrap required, opened a new log
I20260928 23:06:42.446434 17257 ts_tablet_manager.cc:1397] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:42.446437 17256 ts_tablet_manager.cc:1397] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:06:42.447572 17255 raft_consensus.cc:348] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.447774 17255 raft_consensus.cc:374] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:42.447813 17255 raft_consensus.cc:729] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a1cf74389d4bf0a72732a963a0c15e, State: Initialized, Role: FOLLOWER
I20260928 23:06:42.447958 17255 consensus_queue.cc:260] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.448123 17256 raft_consensus.cc:348] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.448239 17255 ts_tablet_manager.cc:1428] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:06:42.448302 17256 raft_consensus.cc:374] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:42.448352 17256 raft_consensus.cc:729] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d991e952e74a4352aca893d908bc80b9, State: Initialized, Role: FOLLOWER
I20260928 23:06:42.448506 17238 heartbeater.cc:503] Master 127.15.212.126:44337 was elected leader, sending a full tablet report...
I20260928 23:06:42.448501 17256 consensus_queue.cc:260] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.448422 17257 raft_consensus.cc:348] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.448599 17257 raft_consensus.cc:374] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:42.448623 17257 raft_consensus.cc:729] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa8a5b05edb446886c76dc3222fc53e, State: Initialized, Role: FOLLOWER
I20260928 23:06:42.448740 17257 consensus_queue.cc:260] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.448762 17256 ts_tablet_manager.cc:1428] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:06:42.448796 17104 heartbeater.cc:503] Master 127.15.212.126:44337 was elected leader, sending a full tablet report...
I20260928 23:06:42.448995 17257 ts_tablet_manager.cc:1428] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260928 23:06:42.449005 16970 heartbeater.cc:503] Master 127.15.212.126:44337 was elected leader, sending a full tablet report...
W20260928 23:06:42.500586 17105 tablet.cc:2369] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:42.509847 17262 raft_consensus.cc:482] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:42.509974 17262 raft_consensus.cc:504] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.510401 17262 leader_election.cc:288] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1a1cf74389d4bf0a72732a963a0c15e (127.15.212.67:46715), 5aa8a5b05edb446886c76dc3222fc53e (127.15.212.65:37641)
I20260928 23:06:42.513736 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.510579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:46715 (local address 127.15.212.66:48461)
0928 23:06:42.510652 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.510666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.510715 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.510904 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.510904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.510906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:42.510994 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.510996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.511804 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.511805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.512756 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.512758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.513421 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.513422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.513551 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.513554 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.513607 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.513638 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 23:06:42.513753 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.510640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:48461 (local address 127.15.212.67:46715)
0928 23:06:42.510691 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.510694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.510706 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:06:42.510789 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.510790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.510820 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.510891 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:42.511081 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.511715 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.512840 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.513341 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.513400 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.513407 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.513637 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.513683 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:06:42.513809 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.510630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:48559 (local address 127.15.212.65:37641)
0928 23:06:42.510692 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.510695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.510964 (+ 269us) server_negotiation.cc:407] Connection header received
0928 23:06:42.511027 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.511029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.511040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.511090 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:42.511235 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.511800 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.512971 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.513443 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.513519 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.513528 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.513699 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.513735 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:06:42.513834 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.510585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:37641 (local address 127.15.212.66:48559)
0928 23:06:42.510880 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:42.510891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:42.510949 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:42.511104 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:42.511105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:42.511106 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:42.511165 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.511167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.511879 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.511879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.512887 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:42.512889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.513524 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.513525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:42.513585 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.513608 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 23:06:42.513678 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 23:06:42.513714 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":93,"thread_start_us":118,"threads_started":1}
I20260928 23:06:42.514127 17192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b80bbf456d64361b3cccead438a59f5" candidate_uuid: "d991e952e74a4352aca893d908bc80b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" is_pre_election: true
I20260928 23:06:42.514156 16924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b80bbf456d64361b3cccead438a59f5" candidate_uuid: "d991e952e74a4352aca893d908bc80b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa8a5b05edb446886c76dc3222fc53e" is_pre_election: true
I20260928 23:06:42.514290 16924 raft_consensus.cc:2454] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d991e952e74a4352aca893d908bc80b9 in term 0.
I20260928 23:06:42.514290 17192 raft_consensus.cc:2454] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d991e952e74a4352aca893d908bc80b9 in term 0.
I20260928 23:06:42.514508 16992 leader_election.cc:302] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa8a5b05edb446886c76dc3222fc53e, d991e952e74a4352aca893d908bc80b9; no voters:
I20260928 23:06:42.514698 17262 raft_consensus.cc:2781] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:42.514755 17262 raft_consensus.cc:482] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:42.514784 17262 raft_consensus.cc:3037] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:42.515466 17262 raft_consensus.cc:504] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.515623 17262 leader_election.cc:288] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [CANDIDATE]: Term 1 election: Requested vote from peers f1a1cf74389d4bf0a72732a963a0c15e (127.15.212.67:46715), 5aa8a5b05edb446886c76dc3222fc53e (127.15.212.65:37641)
I20260928 23:06:42.515779 16924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b80bbf456d64361b3cccead438a59f5" candidate_uuid: "d991e952e74a4352aca893d908bc80b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa8a5b05edb446886c76dc3222fc53e"
I20260928 23:06:42.515822 17192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b80bbf456d64361b3cccead438a59f5" candidate_uuid: "d991e952e74a4352aca893d908bc80b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a1cf74389d4bf0a72732a963a0c15e"
I20260928 23:06:42.515856 16924 raft_consensus.cc:3037] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:42.515919 17192 raft_consensus.cc:3037] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:42.516685 16924 raft_consensus.cc:2454] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d991e952e74a4352aca893d908bc80b9 in term 1.
I20260928 23:06:42.516709 17192 raft_consensus.cc:2454] T 2b80bbf456d64361b3cccead438a59f5 P f1a1cf74389d4bf0a72732a963a0c15e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d991e952e74a4352aca893d908bc80b9 in term 1.
I20260928 23:06:42.516851 16992 leader_election.cc:302] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa8a5b05edb446886c76dc3222fc53e, d991e952e74a4352aca893d908bc80b9; no voters:
I20260928 23:06:42.516950 17262 raft_consensus.cc:2781] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:42.517131 17262 raft_consensus.cc:686] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [term 1 LEADER]: Becoming Leader. State: Replica: d991e952e74a4352aca893d908bc80b9, State: Running, Role: LEADER
I20260928 23:06:42.517242 17262 consensus_queue.cc:237] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } }
I20260928 23:06:42.518121 16779 catalog_manager.cc:5495] T 2b80bbf456d64361b3cccead438a59f5 P d991e952e74a4352aca893d908bc80b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d991e952e74a4352aca893d908bc80b9 (127.15.212.66). New cstate: current_term: 1 leader_uuid: "d991e952e74a4352aca893d908bc80b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d991e952e74a4352aca893d908bc80b9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1a1cf74389d4bf0a72732a963a0c15e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa8a5b05edb446886c76dc3222fc53e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 37641 } health_report { overall_health: UNKNOWN } } }
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:42.540977 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.534102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.15.212.65:37641)
0928 23:06:42.534196 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.534198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.534260 (+ 62us) server_negotiation.cc:407] Connection header received
0928 23:06:42.534305 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.534306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.534315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.534364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.534881 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.535342 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.535999 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.536178 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.536226 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.538511 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.538522 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.538522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.538528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.539929 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.540242 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.540245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.540245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.540296 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.540501 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.540502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.540502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.540638 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.540763 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.540817 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.540878 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:42.562974 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.555974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.15.212.65:37641)
0928 23:06:42.556053 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.556056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.556134 (+ 78us) server_negotiation.cc:407] Connection header received
0928 23:06:42.556235 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.556236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.556246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.556317 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.556943 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.557427 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.558186 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.558385 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.558472 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.560808 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.560809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.560809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.560815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.562032 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.562333 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.562333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.562334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.562381 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.562522 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.562522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.562523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.562608 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.562674 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.562810 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.562869 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[16734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
W20260928 23:06:42.587808 16971 tablet.cc:2369] T 2b80bbf456d64361b3cccead438a59f5 P 5aa8a5b05edb446886c76dc3222fc53e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:42.588749 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:42.580952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.15.212.65:37641)
0928 23:06:42.581021 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:42.581024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:42.581177 (+ 153us) server_negotiation.cc:407] Connection header received
0928 23:06:42.581269 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:42.581270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:42.581287 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:42.581359 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:42.582123 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.582613 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.583648 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:42.583867 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:42.583955 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:42.586184 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:42.586185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:42.586185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:42.586192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:42.587584 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.588010 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.588011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.588012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.588073 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:42.588259 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:42.588260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:42.588260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:42.588349 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:42.588420 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:42.588581 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 23:06:42.588640 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260928 23:06:42.589236 16884 tablet_service.cc:2908] Rejecting scan request for tablet 2b80bbf456d64361b3cccead438a59f5: Uninitialized: clean time has not yet been initialized
I20260928 23:06:42.589979 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16841
I20260928 23:06:42.595377 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16975
I20260928 23:06:42.599898 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17109
I20260928 23:06:42.604137 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 16747
2026-09-28T23:06:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1993 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[17279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[17279](info): set up 2 sockets
Sep 28 23:06:42 dist-test-slave-x2t8 krb5kdc[17279](info): commencing operation
krb5kdc: starting...
W20260928 23:06:43.636078 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:43 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:43.755473 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:43269
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:43269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:43.834570 17292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:43.834810 17292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:43.834844 17292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:43.835984 17292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:43.836055 17292 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 23:06:43.836076 17292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:43.836099 17292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:43.836129 17292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:43.836161 17292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:43.837666 17292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:43269
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:43269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:43.837935 17292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:43.838163 17292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:43.842963 17297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:43.843014 17298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:43.843019 17292 server_base.cc:1034] running on GCE node
W20260928 23:06:43.843000 17300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:43.843380 17292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:43.843662 17292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:43.844843 17292 hybrid_clock.cc:648] HybridClock initialized: now 1790636803844830 us; error 25 us; skew 500 ppm
Sep 28 23:06:43 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636803, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:43.853811 17292 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:43.854254 17292 webserver.cc:460] Webserver started at http://127.15.212.126:46375/ using document root <none> and password file <none>
I20260928 23:06:43.854436 17292 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:43.854476 17292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:43.854564 17292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:43.855587 17292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "f9e8c90dfd7f46fc9672f53d559be447"
format_stamp: "Formatted at 2026-09-28 23:06:43 on dist-test-slave-x2t8"
server_key: "42ef396ddf2d40b998a84c275f598e9c"
server_key_iv: "afdcfe6c3dc2d5805246f576a3cdca4c"
server_key_version: "encryptionkey@0"
I20260928 23:06:43.855891 17292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "f9e8c90dfd7f46fc9672f53d559be447"
format_stamp: "Formatted at 2026-09-28 23:06:43 on dist-test-slave-x2t8"
server_key: "42ef396ddf2d40b998a84c275f598e9c"
server_key_iv: "afdcfe6c3dc2d5805246f576a3cdca4c"
server_key_version: "encryptionkey@0"
I20260928 23:06:43.857245 17292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:43.857947 17307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:43.858155 17292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:43.858244 17292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "f9e8c90dfd7f46fc9672f53d559be447"
format_stamp: "Formatted at 2026-09-28 23:06:43 on dist-test-slave-x2t8"
server_key: "42ef396ddf2d40b998a84c275f598e9c"
server_key_iv: "afdcfe6c3dc2d5805246f576a3cdca4c"
server_key_version: "encryptionkey@0"
I20260928 23:06:43.858299 17292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:43.871775 17292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:43.873749 17292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:43.873853 17292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:43.878396 17292 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:43269
I20260928 23:06:43.878443 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:43269 every 8 connection(s)
I20260928 23:06:43.878763 17292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:43.879354 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:43.879909 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17292
I20260928 23:06:43.879999 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:43.880169 16209 external_mini_cluster.cc:1428] Setting key 68c51347f5076a93b282660d7573a4b6
I20260928 23:06:43.882408 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Bootstrap starting.
I20260928 23:06:43.882977 17370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:43.883209 17370 log.cc:824] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:43.883879 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: No bootstrap required, opened a new log
I20260928 23:06:43.886387 17370 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } }
I20260928 23:06:43.886580 17370 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:43.886619 17370 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e8c90dfd7f46fc9672f53d559be447, State: Initialized, Role: FOLLOWER
I20260928 23:06:43.886756 17370 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } }
I20260928 23:06:43.886834 17370 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:43.886869 17370 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:43.886927 17370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:43.887616 17370 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } }
Sep 28 23:06:43 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:43.887728 17370 leader_election.cc:302] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9e8c90dfd7f46fc9672f53d559be447; no voters:
I20260928 23:06:43.887898 17370 leader_election.cc:288] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:43.887957 17375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:43.888098 17375 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [term 1 LEADER]: Becoming Leader. State: Replica: f9e8c90dfd7f46fc9672f53d559be447, State: Running, Role: LEADER
I20260928 23:06:43.888185 17370 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:43.888219 17375 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } }
I20260928 23:06:43.888511 17375 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9e8c90dfd7f46fc9672f53d559be447. Latest consensus state: current_term: 1 leader_uuid: "f9e8c90dfd7f46fc9672f53d559be447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } } }
I20260928 23:06:43.888594 17375 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:43.888801 17377 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9e8c90dfd7f46fc9672f53d559be447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e8c90dfd7f46fc9672f53d559be447" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43269 } } }
I20260928 23:06:43.888863 17379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:43.888870 17377 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:43.889361 17379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:43.891156 17379 catalog_manager.cc:1347] Generated new cluster ID: 90a5617920be4122b7e20a20b866b6e9
I20260928 23:06:43.891207 17379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:43.892359 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:43.880695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.15.212.126:43269)
0928 23:06:43.880947 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:43.880956 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:43.880998 (+ 42us) server_negotiation.cc:407] Connection header received
0928 23:06:43.881530 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:43.881538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:43.881599 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:43.881751 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:43.882520 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:43.883784 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:43.884870 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:43.885294 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:43.885375 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:43.888230 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:43.888234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:43.888235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:43.888247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:43.890398 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:43.890816 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:43.890819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:43.890820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:43.890912 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:43.891142 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:43.891143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:43.891143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:43.891442 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:43.891568 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:43.892031 (+ 463us) server_negotiation.cc:299] Negotiation successful
0928 23:06:43.892158 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":111,"threads_started":1}
I20260928 23:06:43.920006 17379 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:43.920599 17379 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:43.925460 17379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Generated new TSK 0
I20260928 23:06:43.925669 17379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:43.953406 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43269
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:44.033806 17396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:44.034035 17396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:44.034076 17396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:44.035245 17396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:44.035310 17396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:44.035344 17396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:44.035382 17396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:44.036859 17396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:44.037114 17396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:44.037407 17396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:44.042222 17401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:44.042253 17402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:44.042373 17396 server_base.cc:1034] running on GCE node
W20260928 23:06:44.042268 17404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:44.042687 17396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:44.042948 17396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:44.044085 17396 hybrid_clock.cc:648] HybridClock initialized: now 1790636804044075 us; error 33 us; skew 500 ppm
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:44.046468 17396 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:44.046851 17396 webserver.cc:460] Webserver started at http://127.15.212.65:42137/ using document root <none> and password file <none>
I20260928 23:06:44.047108 17396 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:44.047153 17396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:44.047317 17396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:44.048317 17396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "95da8c4890a8417cae4e6e3fe7b5a8db"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "10dbef0f764bae54ea2c5ab56dbb9d2e"
server_key_iv: "94be3b3d999b630b17196c3e62ea1d09"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.048693 17396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "95da8c4890a8417cae4e6e3fe7b5a8db"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "10dbef0f764bae54ea2c5ab56dbb9d2e"
server_key_iv: "94be3b3d999b630b17196c3e62ea1d09"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.049979 17396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:44.050622 17411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.050787 17396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:44.050879 17396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "95da8c4890a8417cae4e6e3fe7b5a8db"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "10dbef0f764bae54ea2c5ab56dbb9d2e"
server_key_iv: "94be3b3d999b630b17196c3e62ea1d09"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.050956 17396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:44.073793 17396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:44.075860 17396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:44.076032 17396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:44.076681 17396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:44.076740 17396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.076793 17396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:44.076813 17396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:44.083894 17396 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:38641
I20260928 23:06:44.084022 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:38641 every 8 connection(s)
I20260928 23:06:44.084313 17396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:06:44.086728 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.077081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48863 (local address 127.15.212.126:43269)
0928 23:06:44.077159 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.077162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.077456 (+ 294us) server_negotiation.cc:407] Connection header received
0928 23:06:44.077994 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.077996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.078010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.078066 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.078998 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.079883 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.080951 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.081320 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.081397 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.084265 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.084266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.084266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.084276 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.085632 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.086000 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.086001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.086001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.086049 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.086225 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.086226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.086226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.086322 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.086373 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.086598 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.086666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 23:06:44.086922 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.077067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.65:48863)
0928 23:06:44.077268 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.077303 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.077871 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.078139 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.078143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.078563 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.078856 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.078863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.079987 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.079988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.080880 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.080882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.081443 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.081444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.081599 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.082153 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.082161 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.084167 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.085715 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.085720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.085722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.085931 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.086094 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.086095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.086095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.086176 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.086381 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.086385 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.086491 (+ 106us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.086574 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.086686 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260928 23:06:44.087441 17527 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43269
I20260928 23:06:44.087536 17527 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:44.087707 17527 heartbeater.cc:511] Master 127.15.212.126:43269 requested a full tablet report, sending...
I20260928 23:06:44.088336 17323 ts_manager.cc:194] Registered new tserver with Master: 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641)
I20260928 23:06:44.089067 17323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:48863
I20260928 23:06:44.092660 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17396
I20260928 23:06:44.092779 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:44.092947 16209 external_mini_cluster.cc:1428] Setting key 3af1c5255c61847ec006709f4791b704
I20260928 23:06:44.094170 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.089220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:54299 (local address 127.15.212.126:43269)
0928 23:06:44.089277 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.089278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.089287 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:44.089304 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.089305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.089314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.089374 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.089759 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.090228 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.090859 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.091022 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.091068 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.091885 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.091886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.091886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.091890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.093160 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.093526 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.093527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.093527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.093572 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.093751 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.093752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.093752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.093848 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.093899 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.094054 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.094105 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:06:44.094302 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.088501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.65:54299)
0928 23:06:44.088657 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.088668 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.088719 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.089379 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.089380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.089609 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.089682 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.089683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.090310 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.090311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.090792 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.090794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.091083 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.091084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.091134 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.091530 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.091535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.091820 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.093255 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.093256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.093257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.093462 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.093629 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.093630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.093630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.093689 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.093914 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.093915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.093961 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.094040 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.094148 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:44.117443 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43269
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:44.200693 17536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:44.200886 17536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:44.200910 17536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:44.202072 17536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:44.202112 17536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:44.202132 17536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:44.202157 17536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:44.203478 17536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:44.203640 17536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:44.203814 17536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:44.208628 17542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:44.208628 17544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:44.208760 17541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:44.208860 17536 server_base.cc:1034] running on GCE node
I20260928 23:06:44.209137 17536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:44.209379 17536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:44.210551 17536 hybrid_clock.cc:648] HybridClock initialized: now 1790636804210524 us; error 38 us; skew 500 ppm
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:44.213028 17536 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:44.213449 17536 webserver.cc:460] Webserver started at http://127.15.212.66:36539/ using document root <none> and password file <none>
I20260928 23:06:44.213685 17536 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:44.213753 17536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:44.213914 17536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:44.214895 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "60eb19acda434c88b0b16387a264e0d8"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2d3216ebd7cc7c48b168ef0025fccd35"
server_key_iv: "844e556c1d14bd3a69e6cb24fe92003e"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.215255 17536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "60eb19acda434c88b0b16387a264e0d8"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2d3216ebd7cc7c48b168ef0025fccd35"
server_key_iv: "844e556c1d14bd3a69e6cb24fe92003e"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.216609 17536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:06:44.217299 17551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.217458 17536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:44.217564 17536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "60eb19acda434c88b0b16387a264e0d8"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2d3216ebd7cc7c48b168ef0025fccd35"
server_key_iv: "844e556c1d14bd3a69e6cb24fe92003e"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.217651 17536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:44.231467 17536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:44.233850 17536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:44.234011 17536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:44.234570 17536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:44.234642 17536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.234681 17536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:44.234709 17536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:44.241647 17536 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:37999
I20260928 23:06:44.241725 17666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:37999 every 8 connection(s)
I20260928 23:06:44.242076 17536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:44.244834 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.234999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:60937 (local address 127.15.212.126:43269)
0928 23:06:44.235090 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.235092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.235273 (+ 181us) server_negotiation.cc:407] Connection header received
0928 23:06:44.235736 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.235737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.235750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.235832 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.236680 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.237534 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.238623 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.238961 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.239028 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.241763 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.241764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.241765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.241771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.243351 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.243774 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.243776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.243776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.243854 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.244095 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.244096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.244096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.244228 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.244323 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.244664 (+ 341us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.244744 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 23:06:44.245018 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.234977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.66:60937)
0928 23:06:44.235130 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.235178 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.235635 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.235893 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.235896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.236272 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.236571 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.236579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.237719 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.237720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.238518 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.238520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.239073 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.239074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.239243 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.239783 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.239792 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.241620 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.243453 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.243459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.243461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.243688 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.243927 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.243928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.243928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.244028 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.244295 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.244301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.244504 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.244630 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.244797 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260928 23:06:44.245476 17667 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43269
I20260928 23:06:44.245553 17667 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:44.245730 17667 heartbeater.cc:511] Master 127.15.212.126:43269 requested a full tablet report, sending...
I20260928 23:06:44.246117 17323 ts_manager.cc:194] Registered new tserver with Master: 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.246500 17323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:60937
I20260928 23:06:44.251823 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.246538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.66:53659)
0928 23:06:44.246676 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.246685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.246737 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.246880 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.246880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.247045 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.247097 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.247098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.247748 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.247749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.248258 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.248259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.248696 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.248697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.248768 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.249192 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.249196 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.249455 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.250786 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.250787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.250787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.250950 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.251161 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.251162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.251162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.251225 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.251498 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.251499 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.251543 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.251631 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.251723 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260928 23:06:44.251834 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.246584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:53659 (local address 127.15.212.126:43269)
0928 23:06:44.246620 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.246622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.246724 (+ 102us) server_negotiation.cc:407] Connection header received
0928 23:06:44.246815 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.246816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.246825 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.246878 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.247191 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.247656 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.248339 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.248609 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.248694 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.249559 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.249561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.249561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.249567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.250683 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.251035 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.251036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.251037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.251087 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.251290 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.251291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.251291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.251429 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.251505 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.251684 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.251754 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:06:44.252084 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17536
I20260928 23:06:44.252187 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:44.252386 16209 external_mini_cluster.cc:1428] Setting key 07183cc1fde656629b42c52a0fd6e71f
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:44.275239 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43269
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:44.354548 17676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:44.354744 17676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:44.354766 17676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:44.355903 17676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:44.355943 17676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:44.355963 17676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:44.355981 17676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:44.357462 17676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:43269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:44.357714 17676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:44.357946 17676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:44.363061 17682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:44.363061 17681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:44.363078 17676 server_base.cc:1034] running on GCE node
W20260928 23:06:44.363066 17684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:44.363474 17676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:44.363674 17676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:44.364807 17676 hybrid_clock.cc:648] HybridClock initialized: now 1790636804364795 us; error 29 us; skew 500 ppm
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:44.367324 17676 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:44.367748 17676 webserver.cc:460] Webserver started at http://127.15.212.67:36063/ using document root <none> and password file <none>
I20260928 23:06:44.367971 17676 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:44.368019 17676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:44.368189 17676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:44.369305 17676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "16a73ba1375243898c389d0850426e9e"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2eb8af59189ae91250b72cbc62d760b1"
server_key_iv: "11c031dfaca80f79b88b912f9920153d"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.369660 17676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "16a73ba1375243898c389d0850426e9e"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2eb8af59189ae91250b72cbc62d760b1"
server_key_iv: "11c031dfaca80f79b88b912f9920153d"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.370925 17676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:06:44.371603 17691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.371795 17676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:44.371883 17676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "16a73ba1375243898c389d0850426e9e"
format_stamp: "Formatted at 2026-09-28 23:06:44 on dist-test-slave-x2t8"
server_key: "2eb8af59189ae91250b72cbc62d760b1"
server_key_iv: "11c031dfaca80f79b88b912f9920153d"
server_key_version: "encryptionkey@0"
I20260928 23:06:44.371949 17676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:44.392882 17676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:44.395160 17676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:44.395316 17676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:44.395816 17676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:44.395859 17676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.395882 17676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:44.395900 17676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.402572 17676 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:33541
I20260928 23:06:44.402719 17806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:33541 every 8 connection(s)
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:44.402973 17676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:44.405977 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.396370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:58017 (local address 127.15.212.126:43269)
0928 23:06:44.396456 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.396458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.396667 (+ 209us) server_negotiation.cc:407] Connection header received
0928 23:06:44.397160 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.397161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.397173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.397216 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.398128 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.398974 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.400538 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.400865 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.400953 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.403510 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.403511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.403512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.403518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.404771 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.405207 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.405208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.405208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.405266 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.405493 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.405493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.405493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.405595 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.405654 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.405849 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.405915 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 23:06:44.406200 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.396335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.67:58017)
0928 23:06:44.396537 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.396577 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.397054 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.397282 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.397286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.397726 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.398012 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.398019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.399077 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.399078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.400394 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.400397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.400953 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.400954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.401054 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.401569 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.401579 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.403417 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.404841 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.404846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.404847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.405110 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.405336 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.405337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.405337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.405432 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.405650 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.405654 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.405759 (+ 105us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.405839 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.405956 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260928 23:06:44.406672 17807 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43269
I20260928 23:06:44.406751 17807 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:44.406939 17807 heartbeater.cc:511] Master 127.15.212.126:43269 requested a full tablet report, sending...
I20260928 23:06:44.407341 17324 ts_manager.cc:194] Registered new tserver with Master: 16a73ba1375243898c389d0850426e9e (127.15.212.67:33541)
I20260928 23:06:44.407804 17324 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:58017
I20260928 23:06:44.409722 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17676
I20260928 23:06:44.409791 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:44.409964 16209 external_mini_cluster.cc:1428] Setting key 0492857332b0c3387a9d069648fd4a9b
I20260928 23:06:44.410856 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:06:44.413414 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.407788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48571 (local address 127.15.212.126:43269)
0928 23:06:44.407823 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.407824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.408047 (+ 223us) server_negotiation.cc:407] Connection header received
0928 23:06:44.408114 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.408115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.408125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.408177 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.408533 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.408970 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.409631 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.409848 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.409904 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.410854 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.410856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.410856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.410863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.412340 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.412705 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.412706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.412707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.412770 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.412997 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.412998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.412998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.413091 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.413144 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.413307 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.413360 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:06:44.413522 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.407714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.15.212.67:48571)
0928 23:06:44.407972 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.407983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.408041 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.408191 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.408192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.408383 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.408463 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.408465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.409052 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.409052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.409532 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.409553 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.409909 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.409910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.409987 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.410430 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.410434 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.410779 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.412420 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.412421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.412421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.412632 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.412846 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.412847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.412847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.412924 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.413162 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.413163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.413211 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.413291 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.413395 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":60,"threads_started":1}
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:44.433027 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.426570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.15.212.126:43269)
0928 23:06:44.426650 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.426653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.426678 (+ 25us) server_negotiation.cc:407] Connection header received
0928 23:06:44.426707 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.426708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.426716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.426776 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.427318 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.427794 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.428601 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.428783 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.428866 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.430943 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.430944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.430944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.430950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.432038 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.432331 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.432331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.432332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.432379 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.432584 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.432584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.432584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.432692 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.432766 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.432868 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.432921 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 23:06:44.434638 17324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:44.436329 17324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:44.443032 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.438591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:37999 (local address 127.0.0.1:38010)
0928 23:06:44.439007 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.439016 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.439054 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.439228 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.439229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.439231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.439268 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.439269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.440632 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.440633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.441775 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.441777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442714 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.442797 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.442810 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.442886 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.442929 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":42,"thread_start_us":73,"threads_started":1}
I20260928 23:06:44.443219 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.438708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.15.212.67:33541)
0928 23:06:44.438883 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.438889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.438899 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:44.438982 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.438985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.439002 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.439081 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.439398 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.440513 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442004 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442688 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442744 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.442789 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.443072 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.443116 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260928 23:06:44.443266 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.438594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33541 (local address 127.0.0.1:50058)
0928 23:06:44.438777 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.438833 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.438900 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.439162 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.439164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.439166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.439240 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.439242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.440623 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.440624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.441922 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.441925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442821 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.442912 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.442917 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.442968 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.443004 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":48,"thread_start_us":85,"threads_started":1}
I20260928 23:06:44.444111 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.438815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38641 (local address 127.0.0.1:53752)
0928 23:06:44.439384 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.439394 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.439493 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.440764 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.440764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.440767 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.440818 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.440819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442144 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.442954 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.442956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.443872 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.443873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.443943 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.443947 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.444010 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.444050 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":71,"threads_started":1}
I20260928 23:06:44.444120 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.439213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.15.212.65:38641)
0928 23:06:44.440602 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.440607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.440617 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:44.440653 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.440655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.440670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.440751 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.440896 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442042 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.443039 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.443778 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.443867 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.443916 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.444017 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.444046 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1335,"thread_start_us":80,"threads_started":1}
I20260928 23:06:44.444039 17741 tablet_service.cc:1467] Processing CreateTablet for tablet b01d785f2e7d40e8adc719bda4aade8a (DEFAULT_TABLE table=test-table [id=13157ed498ec43baaca865e631a6dd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.444418 17741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01d785f2e7d40e8adc719bda4aade8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.444872 17461 tablet_service.cc:1467] Processing CreateTablet for tablet b01d785f2e7d40e8adc719bda4aade8a (DEFAULT_TABLE table=test-table [id=13157ed498ec43baaca865e631a6dd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.445180 17461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01d785f2e7d40e8adc719bda4aade8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.447718 17830 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Bootstrap starting.
I20260928 23:06:44.443069 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.438912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.15.212.66:37999)
0928 23:06:44.439055 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.439060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.439073 (+ 13us) server_negotiation.cc:407] Connection header received
0928 23:06:44.439118 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.439120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.439136 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.439226 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.439356 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.440547 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.441859 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.442622 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.442704 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.442769 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.442912 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.442961 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260928 23:06:44.448725 17830 tablet_bootstrap.cc:654] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.448701 17601 tablet_service.cc:1467] Processing CreateTablet for tablet b01d785f2e7d40e8adc719bda4aade8a (DEFAULT_TABLE table=test-table [id=13157ed498ec43baaca865e631a6dd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.448745 17831 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Bootstrap starting.
I20260928 23:06:44.449033 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b01d785f2e7d40e8adc719bda4aade8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.449065 17830 log.cc:824] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:44.449450 17831 tablet_bootstrap.cc:654] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.449810 17831 log.cc:824] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:44.449909 17830 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: No bootstrap required, opened a new log
I20260928 23:06:44.449973 17830 ts_tablet_manager.cc:1397] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:44.450632 17831 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: No bootstrap required, opened a new log
I20260928 23:06:44.450770 17831 ts_tablet_manager.cc:1397] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:44.451478 17830 raft_consensus.cc:348] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.451630 17830 raft_consensus.cc:374] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.451673 17830 raft_consensus.cc:729] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a73ba1375243898c389d0850426e9e, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.451797 17830 consensus_queue.cc:260] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.452064 17830 ts_tablet_manager.cc:1428] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:44.452188 17807 heartbeater.cc:503] Master 127.15.212.126:43269 was elected leader, sending a full tablet report...
I20260928 23:06:44.452644 17835 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Bootstrap starting.
I20260928 23:06:44.452980 17831 raft_consensus.cc:348] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.453125 17831 raft_consensus.cc:374] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.453161 17831 raft_consensus.cc:729] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95da8c4890a8417cae4e6e3fe7b5a8db, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.453253 17835 tablet_bootstrap.cc:654] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.453291 17831 consensus_queue.cc:260] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.453567 17831 ts_tablet_manager.cc:1428] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:06:44.453573 17835 log.cc:824] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:44.453619 17527 heartbeater.cc:503] Master 127.15.212.126:43269 was elected leader, sending a full tablet report...
I20260928 23:06:44.454350 17835 tablet_bootstrap.cc:492] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: No bootstrap required, opened a new log
I20260928 23:06:44.454401 17835 ts_tablet_manager.cc:1397] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:44.455689 17835 raft_consensus.cc:348] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.455803 17835 raft_consensus.cc:374] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.455824 17835 raft_consensus.cc:729] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60eb19acda434c88b0b16387a264e0d8, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.455929 17835 consensus_queue.cc:260] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.456130 17835 ts_tablet_manager.cc:1428] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:44.456215 17667 heartbeater.cc:503] Master 127.15.212.126:43269 was elected leader, sending a full tablet report...
I20260928 23:06:44.472960 17836 raft_consensus.cc:482] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:44.473088 17836 raft_consensus.cc:504] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.473414 17836 leader_election.cc:288] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16a73ba1375243898c389d0850426e9e (127.15.212.67:33541), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.477111 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.473676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44987 (local address 127.15.212.67:33541)
0928 23:06:44.473728 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.473730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.473873 (+ 143us) server_negotiation.cc:407] Connection header received
0928 23:06:44.473917 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.473917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.473926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.473963 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.474120 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.474851 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476170 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.476744 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476822 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.476830 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.477008 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.477045 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:06:44.477111 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.473678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:37939 (local address 127.15.212.66:37999)
0928 23:06:44.473720 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.473722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.473734 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:06:44.473759 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.473759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.473768 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.473831 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.474042 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.474774 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476171 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.476747 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476822 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.476830 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.477008 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.477050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:06:44.477151 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.473591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33541 (local address 127.15.212.65:44987)
0928 23:06:44.473778 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.473786 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.473860 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.473983 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.473984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.473986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.474048 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.474050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.474912 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.474912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.476064 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.476067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.476827 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.476922 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.476927 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.476986 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.477035 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":51,"thread_start_us":99,"threads_started":1}
I20260928 23:06:44.477157 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.473591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:37999 (local address 127.15.212.65:37939)
0928 23:06:44.473630 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.473655 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.473695 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.473860 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.473861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.473863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.473930 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.473933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.474859 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.474859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.476064 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.476067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.476827 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.476828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.476930 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.476932 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.476986 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.477035 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":31}
I20260928 23:06:44.477643 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01d785f2e7d40e8adc719bda4aade8a" candidate_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a73ba1375243898c389d0850426e9e" is_pre_election: true
I20260928 23:06:44.477715 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01d785f2e7d40e8adc719bda4aade8a" candidate_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8" is_pre_election: true
I20260928 23:06:44.477800 17761 raft_consensus.cc:2454] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95da8c4890a8417cae4e6e3fe7b5a8db in term 0.
I20260928 23:06:44.477861 17621 raft_consensus.cc:2454] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95da8c4890a8417cae4e6e3fe7b5a8db in term 0.
I20260928 23:06:44.478021 17414 leader_election.cc:302] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a73ba1375243898c389d0850426e9e, 95da8c4890a8417cae4e6e3fe7b5a8db; no voters:
I20260928 23:06:44.478127 17836 raft_consensus.cc:2781] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:44.478201 17836 raft_consensus.cc:482] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:44.478236 17836 raft_consensus.cc:3037] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.479110 17836 raft_consensus.cc:504] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.479254 17836 leader_election.cc:288] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [CANDIDATE]: Term 1 election: Requested vote from peers 16a73ba1375243898c389d0850426e9e (127.15.212.67:33541), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.479408 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01d785f2e7d40e8adc719bda4aade8a" candidate_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8"
I20260928 23:06:44.479409 17761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b01d785f2e7d40e8adc719bda4aade8a" candidate_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a73ba1375243898c389d0850426e9e"
I20260928 23:06:44.479501 17621 raft_consensus.cc:3037] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.479497 17761 raft_consensus.cc:3037] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.480146 17621 raft_consensus.cc:2454] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95da8c4890a8417cae4e6e3fe7b5a8db in term 1.
I20260928 23:06:44.480206 17761 raft_consensus.cc:2454] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95da8c4890a8417cae4e6e3fe7b5a8db in term 1.
I20260928 23:06:44.480352 17413 leader_election.cc:302] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60eb19acda434c88b0b16387a264e0d8, 95da8c4890a8417cae4e6e3fe7b5a8db; no voters:
I20260928 23:06:44.480465 17836 raft_consensus.cc:2781] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:44.480594 17836 raft_consensus.cc:686] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 LEADER]: Becoming Leader. State: Replica: 95da8c4890a8417cae4e6e3fe7b5a8db, State: Running, Role: LEADER
I20260928 23:06:44.480681 17836 consensus_queue.cc:237] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } }
I20260928 23:06:44.481405 17324 catalog_manager.cc:5495] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65). New cstate: current_term: 1 leader_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } health_report { overall_health: HEALTHY } } }
I20260928 23:06:44.491699 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.489507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.15.212.65:38641)
0928 23:06:44.489571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.489573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.489582 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:44.489646 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.489646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.489659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.489716 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:44.489845 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.490293 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.491002 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.491182 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.491237 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.491415 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.491495 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.491587 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.491624 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.491639 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260928 23:06:44.492550 17668 tablet.cc:2369] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:44.493191 17621 raft_consensus.cc:1261] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Refusing update from remote peer 95da8c4890a8417cae4e6e3fe7b5a8db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.493191 17761 raft_consensus.cc:1261] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Refusing update from remote peer 95da8c4890a8417cae4e6e3fe7b5a8db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.493458 17836 consensus_queue.cc:1035] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.493605 17840 consensus_queue.cc:1035] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.495806 17843 mvcc.cc:205] Tried to move back new op lower bound from 7334448351201923072 to 7334448351153364992. Current Snapshot: MvccSnapshot[applied={T|T < 7334448351201923072}]
I20260928 23:06:44.496229 17845 mvcc.cc:205] Tried to move back new op lower bound from 7334448351201923072 to 7334448351153364992. Current Snapshot: MvccSnapshot[applied={T|T < 7334448351201923072}]
I20260928 23:06:44.496635 17844 mvcc.cc:205] Tried to move back new op lower bound from 7334448351201923072 to 7334448351153364992. Current Snapshot: MvccSnapshot[applied={T|T < 7334448351201923072}]
I20260928 23:06:44.499312 17324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:44.499383 17324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:44.500074 17324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Sending DeleteTablet for 3 replicas of tablet b01d785f2e7d40e8adc719bda4aade8a
I20260928 23:06:44.500422 17461 tablet_service.cc:1514] Processing DeleteTablet for tablet b01d785f2e7d40e8adc719bda4aade8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53752
I20260928 23:06:44.500455 17741 tablet_service.cc:1514] Processing DeleteTablet for tablet b01d785f2e7d40e8adc719bda4aade8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:50058
I20260928 23:06:44.500438 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet b01d785f2e7d40e8adc719bda4aade8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:38010
I20260928 23:06:44.500702 17854 tablet_replica.cc:330] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: stopping tablet replica
I20260928 23:06:44.500828 17854 raft_consensus.cc:2229] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:44.500860 17324 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 23:06:44.501015 17852 tablet_replica.cc:330] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: stopping tablet replica
W20260928 23:06:44.501016 17324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:44.501089 17852 raft_consensus.cc:2229] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:44.501204 17852 raft_consensus.cc:2258] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.501210 17854 raft_consensus.cc:2258] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.501761 17854 ts_tablet_manager.cc:1905] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.502059 17461 tablet_service.cc:1467] Processing CreateTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d (DEFAULT_TABLE table=test-table [id=4d24e473197246429ff244fa36e97828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.502070 17741 tablet_service.cc:1467] Processing CreateTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d (DEFAULT_TABLE table=test-table [id=4d24e473197246429ff244fa36e97828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.502130 17852 ts_tablet_manager.cc:1905] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.502162 17461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0cd44cffebe4ad5b922b49eda802c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.502170 17741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0cd44cffebe4ad5b922b49eda802c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.503480 17831 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Bootstrap starting.
I20260928 23:06:44.503482 17830 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Bootstrap starting.
I20260928 23:06:44.503870 17831 tablet_bootstrap.cc:654] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.503961 17852 ts_tablet_manager.cc:1918] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:06:44.504022 17852 log.cc:1196] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/b01d785f2e7d40e8adc719bda4aade8a
I20260928 23:06:44.504048 17830 tablet_bootstrap.cc:654] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.504287 17852 ts_tablet_manager.cc:1939] T b01d785f2e7d40e8adc719bda4aade8a P 60eb19acda434c88b0b16387a264e0d8: Deleting consensus metadata
I20260928 23:06:44.504401 17601 tablet_service.cc:1467] Processing CreateTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d (DEFAULT_TABLE table=test-table [id=4d24e473197246429ff244fa36e97828]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:44.504518 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0cd44cffebe4ad5b922b49eda802c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.504614 17853 tablet_replica.cc:330] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: stopping tablet replica
I20260928 23:06:44.504699 17853 raft_consensus.cc:2229] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:44.504792 17853 raft_consensus.cc:2258] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.505204 17830 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: No bootstrap required, opened a new log
I20260928 23:06:44.505246 17830 ts_tablet_manager.cc:1397] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:44.505271 17853 ts_tablet_manager.cc:1905] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.505388 17830 raft_consensus.cc:348] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.505462 17830 raft_consensus.cc:374] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.505487 17830 raft_consensus.cc:729] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a73ba1375243898c389d0850426e9e, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.505539 17830 consensus_queue.cc:260] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.505586 17831 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: No bootstrap required, opened a new log
I20260928 23:06:44.505622 17831 ts_tablet_manager.cc:1397] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:44.505623 17830 ts_tablet_manager.cc:1428] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:44.505703 17835 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Bootstrap starting.
I20260928 23:06:44.505736 17831 raft_consensus.cc:348] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.505796 17831 raft_consensus.cc:374] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.505810 17831 raft_consensus.cc:729] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95da8c4890a8417cae4e6e3fe7b5a8db, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.505868 17831 consensus_queue.cc:260] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.505949 17831 ts_tablet_manager.cc:1428] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.506093 17835 tablet_bootstrap.cc:654] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.506749 17309 catalog_manager.cc:4841] TS 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999): tablet b01d785f2e7d40e8adc719bda4aade8a (table test-table [id=13157ed498ec43baaca865e631a6dd26]) successfully deleted
I20260928 23:06:44.506861 17853 ts_tablet_manager.cc:1918] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:06:44.506916 17853 log.cc:1196] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/b01d785f2e7d40e8adc719bda4aade8a
I20260928 23:06:44.506928 17835 tablet_bootstrap.cc:492] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: No bootstrap required, opened a new log
I20260928 23:06:44.506959 17835 ts_tablet_manager.cc:1397] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:44.507067 17835 raft_consensus.cc:348] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.507107 17835 raft_consensus.cc:374] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.507128 17835 raft_consensus.cc:729] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60eb19acda434c88b0b16387a264e0d8, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.507164 17853 ts_tablet_manager.cc:1939] T b01d785f2e7d40e8adc719bda4aade8a P 16a73ba1375243898c389d0850426e9e: Deleting consensus metadata
I20260928 23:06:44.507205 17835 consensus_queue.cc:260] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.507334 17835 ts_tablet_manager.cc:1428] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.507542 17310 catalog_manager.cc:4841] TS 16a73ba1375243898c389d0850426e9e (127.15.212.67:33541): tablet b01d785f2e7d40e8adc719bda4aade8a (table test-table [id=13157ed498ec43baaca865e631a6dd26]) successfully deleted
I20260928 23:06:44.508806 17854 ts_tablet_manager.cc:1918] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:06:44.508879 17854 log.cc:1196] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/b01d785f2e7d40e8adc719bda4aade8a
I20260928 23:06:44.509125 17854 ts_tablet_manager.cc:1939] T b01d785f2e7d40e8adc719bda4aade8a P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting consensus metadata
I20260928 23:06:44.509469 17309 catalog_manager.cc:4841] TS 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641): tablet b01d785f2e7d40e8adc719bda4aade8a (table test-table [id=13157ed498ec43baaca865e631a6dd26]) successfully deleted
I20260928 23:06:44.518811 17834 raft_consensus.cc:482] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:44.518891 17834 raft_consensus.cc:504] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.519165 17834 leader_election.cc:288] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.522403 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.519530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48883 (local address 127.15.212.66:37999)
0928 23:06:44.519574 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.519576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.519601 (+ 25us) server_negotiation.cc:407] Connection header received
0928 23:06:44.519686 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.519687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.519697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.519727 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.519904 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.520544 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.521677 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.522083 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.522155 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.522163 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.522308 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.522341 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:06:44.522418 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.519438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:37999 (local address 127.15.212.67:48883)
0928 23:06:44.519553 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.519563 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.519612 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.519784 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.519785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.519787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.519838 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.519839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.520627 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.520628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.521590 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.521596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.522162 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.522163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.522235 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.522240 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.522297 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.522336 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260928 23:06:44.522953 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.519323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38641 (local address 127.15.212.67:60133)
0928 23:06:44.519375 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.519388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.519442 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.519656 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.519657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.519659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:44.519729 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.519731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.520741 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.520742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.521937 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.521940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.522670 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.522671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.522763 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.522784 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.522837 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.522877 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 23:06:44.522953 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0cd44cffebe4ad5b922b49eda802c0d" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8" is_pre_election: true
I20260928 23:06:44.523029 17621 raft_consensus.cc:2454] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 0.
I20260928 23:06:44.523151 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.519387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:60133 (local address 127.15.212.65:38641)
0928 23:06:44.519426 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.519428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.519437 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:44.519575 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.519576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.519585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.519645 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:44.519797 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.520576 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.522026 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.522582 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.522671 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.522678 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.523040 (+ 362us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.523082 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:06:44.523206 17693 leader_election.cc:302] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a73ba1375243898c389d0850426e9e, 60eb19acda434c88b0b16387a264e0d8; no voters:
I20260928 23:06:44.523412 17834 raft_consensus.cc:2781] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:44.523454 17834 raft_consensus.cc:482] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:44.523479 17834 raft_consensus.cc:3037] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.523473 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0cd44cffebe4ad5b922b49eda802c0d" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" is_pre_election: true
I20260928 23:06:44.523591 17481 raft_consensus.cc:2454] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 0.
I20260928 23:06:44.524282 17834 raft_consensus.cc:504] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.524405 17834 leader_election.cc:288] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 election: Requested vote from peers 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.524541 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0cd44cffebe4ad5b922b49eda802c0d" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db"
I20260928 23:06:44.524572 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0cd44cffebe4ad5b922b49eda802c0d" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8"
I20260928 23:06:44.524613 17481 raft_consensus.cc:3037] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.524637 17621 raft_consensus.cc:3037] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.525323 17621 raft_consensus.cc:2454] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 1.
I20260928 23:06:44.525378 17481 raft_consensus.cc:2454] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 1.
I20260928 23:06:44.525475 17693 leader_election.cc:302] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a73ba1375243898c389d0850426e9e, 60eb19acda434c88b0b16387a264e0d8; no voters:
I20260928 23:06:44.525574 17834 raft_consensus.cc:2781] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:44.525693 17834 raft_consensus.cc:686] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 1 LEADER]: Becoming Leader. State: Replica: 16a73ba1375243898c389d0850426e9e, State: Running, Role: LEADER
I20260928 23:06:44.525763 17834 consensus_queue.cc:237] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.526413 17323 catalog_manager.cc:5495] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16a73ba1375243898c389d0850426e9e (127.15.212.67). New cstate: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } health_report { overall_health: HEALTHY } } }
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636803, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:44.556939 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.549214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.15.212.126:43269)
0928 23:06:44.549282 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.549284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.549413 (+ 129us) server_negotiation.cc:407] Connection header received
0928 23:06:44.549480 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.549480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.549488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.549537 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.550103 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.550767 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.551800 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.551977 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.552025 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.554349 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.554350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.554350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.554357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.555722 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.556122 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.556123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.556124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.556180 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.556366 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.556367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.556368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.556530 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.556603 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.556749 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.556829 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:06:44.557008 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.549161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43269 (local address 127.0.0.1:45676)
0928 23:06:44.549333 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.549345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.549418 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.549564 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.549565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.549957 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:44.550026 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.550028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.550841 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.550842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.551708 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.551710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.552046 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.552047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.552107 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.552689 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:44.552694 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:44.554236 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:44.555840 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.555846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.555848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.556036 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.556247 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:44.556248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:44.556248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:44.556303 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:44.556594 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:44.556600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:44.556644 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.556726 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.556883 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":64,"thread_start_us":100,"threads_started":1}
I20260928 23:06:44.559110 17323 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:45676:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 23:06:44.559351 17323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:44.560767 17741 tablet_service.cc:1467] Processing CreateTablet for tablet fdaec809898347c18cbdb3e083c288ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=410a829a3935478985a84f2a7fb72312]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:44.560801 17461 tablet_service.cc:1467] Processing CreateTablet for tablet fdaec809898347c18cbdb3e083c288ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=410a829a3935478985a84f2a7fb72312]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:44.560933 17461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaec809898347c18cbdb3e083c288ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.560923 17741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaec809898347c18cbdb3e083c288ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.562158 17831 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: Bootstrap starting.
I20260928 23:06:44.562548 17830 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: Bootstrap starting.
I20260928 23:06:44.562573 17831 tablet_bootstrap.cc:654] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.562651 17601 tablet_service.cc:1467] Processing CreateTablet for tablet fdaec809898347c18cbdb3e083c288ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=410a829a3935478985a84f2a7fb72312]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:44.562804 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdaec809898347c18cbdb3e083c288ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:44.562956 17830 tablet_bootstrap.cc:654] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.563551 17830 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: No bootstrap required, opened a new log
I20260928 23:06:44.563582 17830 ts_tablet_manager.cc:1397] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:44.563696 17830 raft_consensus.cc:348] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.563742 17830 raft_consensus.cc:374] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.563755 17830 raft_consensus.cc:729] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a73ba1375243898c389d0850426e9e, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.563776 17830 consensus_queue.cc:260] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.563870 17830 ts_tablet_manager.cc:1428] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:44.563908 17834 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.563964 17834 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.564199 17835 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: Bootstrap starting.
I20260928 23:06:44.564621 17835 tablet_bootstrap.cc:654] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:44.565209 17831 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: No bootstrap required, opened a new log
I20260928 23:06:44.565264 17831 ts_tablet_manager.cc:1397] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 23:06:44.565383 17835 tablet_bootstrap.cc:492] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: No bootstrap required, opened a new log
I20260928 23:06:44.565436 17835 ts_tablet_manager.cc:1397] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:44.565407 17831 raft_consensus.cc:348] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.565492 17831 raft_consensus.cc:374] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.565517 17831 raft_consensus.cc:729] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95da8c4890a8417cae4e6e3fe7b5a8db, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.565562 17831 consensus_queue.cc:260] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.565649 17831 ts_tablet_manager.cc:1428] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:44.565618 17835 raft_consensus.cc:348] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.565681 17835 raft_consensus.cc:374] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:44.565711 17835 raft_consensus.cc:729] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60eb19acda434c88b0b16387a264e0d8, State: Initialized, Role: FOLLOWER
I20260928 23:06:44.565747 17835 consensus_queue.cc:260] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.565752 17836 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.565820 17836 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.565829 17835 ts_tablet_manager.cc:1428] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:44.566015 17838 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.566112 17838 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260928 23:06:44.585438 17528 tablet.cc:2369] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:06:44.653532 17808 tablet.cc:2369] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:44.793272 17834 raft_consensus.cc:482] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:44.793358 17834 raft_consensus.cc:504] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.793505 17834 leader_election.cc:288] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.793708 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaec809898347c18cbdb3e083c288ef" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" is_pre_election: true
I20260928 23:06:44.793802 17481 raft_consensus.cc:2454] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 0.
I20260928 23:06:44.793780 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaec809898347c18cbdb3e083c288ef" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8" is_pre_election: true
I20260928 23:06:44.793895 17621 raft_consensus.cc:2454] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 0.
I20260928 23:06:44.793967 17693 leader_election.cc:302] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a73ba1375243898c389d0850426e9e, 95da8c4890a8417cae4e6e3fe7b5a8db; no voters:
I20260928 23:06:44.794076 17834 raft_consensus.cc:2781] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:44.794116 17834 raft_consensus.cc:482] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:44.794137 17834 raft_consensus.cc:3037] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.794834 17834 raft_consensus.cc:504] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.794960 17834 leader_election.cc:288] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 election: Requested vote from peers 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641), 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999)
I20260928 23:06:44.795176 17621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaec809898347c18cbdb3e083c288ef" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60eb19acda434c88b0b16387a264e0d8"
I20260928 23:06:44.795171 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaec809898347c18cbdb3e083c288ef" candidate_uuid: "16a73ba1375243898c389d0850426e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db"
I20260928 23:06:44.795269 17481 raft_consensus.cc:3037] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.795269 17621 raft_consensus.cc:3037] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:44.795989 17621 raft_consensus.cc:2454] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 1.
I20260928 23:06:44.795989 17481 raft_consensus.cc:2454] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16a73ba1375243898c389d0850426e9e in term 1.
I20260928 23:06:44.796154 17693 leader_election.cc:302] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a73ba1375243898c389d0850426e9e, 60eb19acda434c88b0b16387a264e0d8; no voters:
I20260928 23:06:44.796247 17834 raft_consensus.cc:2781] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:44.796298 17834 raft_consensus.cc:686] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [term 1 LEADER]: Becoming Leader. State: Replica: 16a73ba1375243898c389d0850426e9e, State: Running, Role: LEADER
I20260928 23:06:44.796343 17834 consensus_queue.cc:237] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } }
I20260928 23:06:44.796654 17859 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: TxnStatusTablet state changed. Reason: New leader 16a73ba1375243898c389d0850426e9e. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.796754 17859 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:44.796957 17871 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 23:06:44.797101 17871 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 23:06:44.796981 17323 catalog_manager.cc:5495] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16a73ba1375243898c389d0850426e9e (127.15.212.67). New cstate: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } health_report { overall_health: HEALTHY } } }
I20260928 23:06:44.849839 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.847018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33541 (local address 127.0.0.1:50068)
0928 23:06:44.847142 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.847153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.847201 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.847362 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.847363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.847364 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:44.847406 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.847408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.848307 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.848307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.848913 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.848915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.849369 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.849370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.849414 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.849425 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.849711 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.849717 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.849751 (+ 34us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.849785 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260928 23:06:44.849852 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.847088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.15.212.67:33541)
0928 23:06:44.847135 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.847137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.847186 (+ 49us) server_negotiation.cc:407] Connection header received
0928 23:06:44.847268 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.847269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.847287 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.847347 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:44.847472 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.848232 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.848994 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.849290 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.849363 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.849514 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.849639 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.849716 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.849755 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.849774 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:06:44.851027 17481 raft_consensus.cc:1261] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Refusing update from remote peer 16a73ba1375243898c389d0850426e9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.851258 17621 raft_consensus.cc:1261] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Refusing update from remote peer 16a73ba1375243898c389d0850426e9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.851359 17834 consensus_queue.cc:1035] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.851486 17859 consensus_queue.cc:1035] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.852075 17838 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: TxnStatusTablet state changed. Reason: New leader 16a73ba1375243898c389d0850426e9e. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.852173 17838 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.852123 17836 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: TxnStatusTablet state changed. Reason: New leader 16a73ba1375243898c389d0850426e9e. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.852238 17836 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.853001 17834 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.853084 17834 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:44.853206 17836 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.853291 17836 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 95da8c4890a8417cae4e6e3fe7b5a8db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.853976 17859 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.854054 17859 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 16a73ba1375243898c389d0850426e9e: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:44.854472 17838 tablet_replica.cc:439] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16a73ba1375243898c389d0850426e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 } } peers { permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 } } peers { permanent_uuid: "16a73ba1375243898c389d0850426e9e" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33541 } } }
I20260928 23:06:44.854568 17838 tablet_replica.cc:442] T fdaec809898347c18cbdb3e083c288ef P 60eb19acda434c88b0b16387a264e0d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:44.857393 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.855071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.15.212.67:33541)
0928 23:06:44.855125 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.855127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.855147 (+ 20us) server_negotiation.cc:407] Connection header received
0928 23:06:44.855200 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.855201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.855207 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.855270 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:44.855418 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.855886 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.856729 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.856931 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.857012 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.857166 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.857195 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.857246 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.857308 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.857325 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:06:44.861876 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.859548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33541 (local address 127.15.212.67:40689)
0928 23:06:44.859624 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:44.859632 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:44.859664 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:44.859804 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:44.859805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:44.859807 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:44.859836 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.859837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.860393 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.860394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.861086 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:44.861087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.861436 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.861437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:44.861485 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.861498 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.861727 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.861728 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:06:44.861773 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 23:06:44.861810 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 23:06:44.861876 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.859574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:40689 (local address 127.15.212.67:33541)
0928 23:06:44.859707 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.859708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.859712 (+ 4us) server_negotiation.cc:407] Connection header received
0928 23:06:44.859724 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.859725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.859730 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.859786 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:44.859881 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.860303 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.861182 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.861353 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.861435 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.861593 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:44.861626 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:44.861702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.861788 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.861803 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260928 23:06:44.863739 17481 raft_consensus.cc:1261] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Refusing update from remote peer 16a73ba1375243898c389d0850426e9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.863772 17621 raft_consensus.cc:1261] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Refusing update from remote peer 16a73ba1375243898c389d0850426e9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:44.864007 17834 consensus_queue.cc:1035] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60eb19acda434c88b0b16387a264e0d8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.864131 17878 consensus_queue.cc:1035] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "95da8c4890a8417cae4e6e3fe7b5a8db" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:44.876132 17323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:44.876217 17323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:44.876787 17323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9e8c90dfd7f46fc9672f53d559be447: Sending DeleteTablet for 3 replicas of tablet c0cd44cffebe4ad5b922b49eda802c0d
I20260928 23:06:44.877029 17461 tablet_service.cc:1514] Processing DeleteTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53752
I20260928 23:06:44.877133 17854 tablet_replica.cc:330] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: stopping tablet replica
I20260928 23:06:44.877125 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:38010
I20260928 23:06:44.877192 17854 raft_consensus.cc:2229] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:44.877228 17854 raft_consensus.cc:2258] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.877235 17852 tablet_replica.cc:330] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: stopping tablet replica
I20260928 23:06:44.877274 17740 tablet_service.cc:1514] Processing DeleteTablet for tablet c0cd44cffebe4ad5b922b49eda802c0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:44 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:50058
I20260928 23:06:44.877291 17852 raft_consensus.cc:2229] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:44.877324 17852 raft_consensus.cc:2258] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.877342 17853 tablet_replica.cc:330] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: stopping tablet replica
I20260928 23:06:44.877379 17853 raft_consensus.cc:2229] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:44.877451 17853 raft_consensus.cc:2258] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:44.877787 17853 ts_tablet_manager.cc:1905] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.877880 17854 ts_tablet_manager.cc:1905] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.879518 17852 ts_tablet_manager.cc:1905] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:44.879578 17854 ts_tablet_manager.cc:1918] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:44.879626 17854 log.cc:1196] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/c0cd44cffebe4ad5b922b49eda802c0d
I20260928 23:06:44.879880 17854 ts_tablet_manager.cc:1939] T c0cd44cffebe4ad5b922b49eda802c0d P 95da8c4890a8417cae4e6e3fe7b5a8db: Deleting consensus metadata
I20260928 23:06:44.880141 17309 catalog_manager.cc:4841] TS 95da8c4890a8417cae4e6e3fe7b5a8db (127.15.212.65:38641): tablet c0cd44cffebe4ad5b922b49eda802c0d (table test-table [id=4d24e473197246429ff244fa36e97828]) successfully deleted
I20260928 23:06:44.881033 17852 ts_tablet_manager.cc:1918] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:44.881081 17852 log.cc:1196] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/c0cd44cffebe4ad5b922b49eda802c0d
I20260928 23:06:44.881284 17852 ts_tablet_manager.cc:1939] T c0cd44cffebe4ad5b922b49eda802c0d P 60eb19acda434c88b0b16387a264e0d8: Deleting consensus metadata
I20260928 23:06:44.881450 17853 ts_tablet_manager.cc:1918] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:44.881506 17853 log.cc:1196] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/c0cd44cffebe4ad5b922b49eda802c0d
I20260928 23:06:44.881711 17853 ts_tablet_manager.cc:1939] T c0cd44cffebe4ad5b922b49eda802c0d P 16a73ba1375243898c389d0850426e9e: Deleting consensus metadata
I20260928 23:06:44.881989 17310 catalog_manager.cc:4841] TS 16a73ba1375243898c389d0850426e9e (127.15.212.67:33541): tablet c0cd44cffebe4ad5b922b49eda802c0d (table test-table [id=4d24e473197246429ff244fa36e97828]) successfully deleted
I20260928 23:06:44.882107 17309 catalog_manager.cc:4841] TS 60eb19acda434c88b0b16387a264e0d8 (127.15.212.66:37999): tablet c0cd44cffebe4ad5b922b49eda802c0d (table test-table [id=4d24e473197246429ff244fa36e97828]) successfully deleted
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636804, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:44.889200 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.877996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.15.212.65:38641)
0928 23:06:44.879503 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.879505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.880318 (+ 813us) server_negotiation.cc:407] Connection header received
0928 23:06:44.880411 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.880411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.880421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.880479 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.881221 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.881953 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.882748 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.883042 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.883120 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.886022 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.886030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.886031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.886038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.887935 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.888269 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.888272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.888273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.888339 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.888639 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.888640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.888640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.888771 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.889010 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.889052 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.889135 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1486}
W20260928 23:06:44.889425 17521 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53762
I20260928 23:06:44.895954 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:44.890367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.15.212.126:43269)
0928 23:06:44.890409 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:44.890410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:44.890444 (+ 34us) server_negotiation.cc:407] Connection header received
0928 23:06:44.890524 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:44.890524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:44.890534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:44.890599 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:44.891076 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.891583 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.892276 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:44.892533 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:44.892611 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:44.893636 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:44.893645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:44.893645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:44.893652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:44.894890 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.895232 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.895233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.895234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.895298 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:44.895495 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:44.895496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:44.895497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:44.895602 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:44.895706 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:44.895796 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 23:06:44.895865 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 23:06:44.896162 17323 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45678
I20260928 23:06:44.896843 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17396
I20260928 23:06:44.903353 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17536
I20260928 23:06:44.908345 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17676
I20260928 23:06:44.913615 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17292
2026-09-28T23:06:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2308 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17889](info): set up 2 sockets
Sep 28 23:06:44 dist-test-slave-x2t8 krb5kdc[17889](info): commencing operation
krb5kdc: starting...
W20260928 23:06:45.940088 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:46.051497 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46345
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:46.130460 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:46.130716 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:46.130750 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:46.132575 17902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:46.132647 17902 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 23:06:46.132665 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:46.132683 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:46.132711 17902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:46.132741 17902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:46.134248 17902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46345
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:46.134421 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:46.134720 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:46.139436 17908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.139463 17910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.139503 17907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.139771 17902 server_base.cc:1034] running on GCE node
I20260928 23:06:46.140002 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:46.140282 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:46.141435 17902 hybrid_clock.cc:648] HybridClock initialized: now 1790636806141406 us; error 41 us; skew 500 ppm
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:46.143597 17902 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:46.143987 17902 webserver.cc:460] Webserver started at http://127.15.212.126:33473/ using document root <none> and password file <none>
I20260928 23:06:46.144201 17902 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:46.144266 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:46.144413 17902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:46.145809 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "b2df0af5355b4b4dae3102f9dea6f234"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "6b162cc62dd68b6d4bfea9cb5194a4a1"
server_key_iv: "b7731e05682e99e1bcf4ff06f9832f9c"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.146159 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "b2df0af5355b4b4dae3102f9dea6f234"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "6b162cc62dd68b6d4bfea9cb5194a4a1"
server_key_iv: "b7731e05682e99e1bcf4ff06f9832f9c"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.147403 17902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:46.148082 17917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.148267 17902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:46.148360 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "b2df0af5355b4b4dae3102f9dea6f234"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "6b162cc62dd68b6d4bfea9cb5194a4a1"
server_key_iv: "b7731e05682e99e1bcf4ff06f9832f9c"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.148443 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:46.155930 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:46.157627 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:46.157773 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:46.162094 17902 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46345
I20260928 23:06:46.162192 17979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46345 every 8 connection(s)
I20260928 23:06:46.162473 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:46.163091 17980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:46.165254 17980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Bootstrap starting.
I20260928 23:06:46.165556 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 17902
I20260928 23:06:46.165663 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:46.165824 16209 external_mini_cluster.cc:1428] Setting key 413c06ec07fca14761d483e17bbe8e8b
I20260928 23:06:46.165858 17980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:46.166193 17980 log.cc:824] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:46.166985 17980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: No bootstrap required, opened a new log
I20260928 23:06:46.168823 17980 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } }
I20260928 23:06:46.168979 17980 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:46.169013 17980 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2df0af5355b4b4dae3102f9dea6f234, State: Initialized, Role: FOLLOWER
I20260928 23:06:46.169127 17980 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } }
I20260928 23:06:46.169198 17980 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:46.169225 17980 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:46.169266 17980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:46.169869 17980 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } }
I20260928 23:06:46.169952 17980 leader_election.cc:302] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2df0af5355b4b4dae3102f9dea6f234; no voters:
I20260928 23:06:46.170127 17980 leader_election.cc:288] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:46.170149 17985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:46.170260 17985 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [term 1 LEADER]: Becoming Leader. State: Replica: b2df0af5355b4b4dae3102f9dea6f234, State: Running, Role: LEADER
I20260928 23:06:46.170336 17985 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } }
I20260928 23:06:46.170365 17980 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:46.170614 17985 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2df0af5355b4b4dae3102f9dea6f234. Latest consensus state: current_term: 1 leader_uuid: "b2df0af5355b4b4dae3102f9dea6f234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } } }
I20260928 23:06:46.170692 17985 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:46.170626 17987 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2df0af5355b4b4dae3102f9dea6f234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2df0af5355b4b4dae3102f9dea6f234" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46345 } } }
I20260928 23:06:46.170858 17987 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234 [sys.catalog]: This master's current role is: LEADER
W20260928 23:06:46.172091 18001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:06:46.172144 18001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:06:46.172201 17990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:46.172333 17990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:46.173804 17990 catalog_manager.cc:1347] Generated new cluster ID: 6a0254e521df460faf00829d99b5adfe
I20260928 23:06:46.173858 17990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:46.177403 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.166296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59574 (local address 127.15.212.126:46345)
0928 23:06:46.166462 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.166471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.166561 (+ 90us) server_negotiation.cc:407] Connection header received
0928 23:06:46.167190 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.167198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.167256 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.167403 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.167944 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.169690 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.170709 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.171271 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.171353 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.174159 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.174163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.174163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.174174 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.175829 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.176252 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.176255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.176255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.176333 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.176577 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.176578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.176578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.176803 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.176964 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.177080 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.177215 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260928 23:06:46.193188 17990 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:46.193857 17990 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:46.206274 17990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Generated new TSK 0
I20260928 23:06:46.206436 17990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:46.235605 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46345
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:46.315094 18006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:46.315297 18006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:46.315322 18006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:46.316478 18006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:46.316519 18006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:46.316540 18006 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 23:06:46.316591 18006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:46.316619 18006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:46.318080 18006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:46345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:46.318334 18006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:46.318560 18006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:46.323578 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.323597 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.323760 18006 server_base.cc:1034] running on GCE node
W20260928 23:06:46.323601 18014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.324069 18006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:46.324290 18006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:46.325426 18006 hybrid_clock.cc:648] HybridClock initialized: now 1790636806325409 us; error 30 us; skew 500 ppm
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:46.327848 18006 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:46.328230 18006 webserver.cc:460] Webserver started at http://127.15.212.65:42219/ using document root <none> and password file <none>
I20260928 23:06:46.328420 18006 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:46.328521 18006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:46.328615 18006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:46.329576 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "fb0509b3a73646c893e1e51a7a1b950f"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "33bb638585950823534db3a6d726b8bf"
server_key_iv: "e39359b793973375736626e95e9f786f"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.329941 18006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "fb0509b3a73646c893e1e51a7a1b950f"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "33bb638585950823534db3a6d726b8bf"
server_key_iv: "e39359b793973375736626e95e9f786f"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.331178 18006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:46.331818 18021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.331990 18006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:46.332091 18006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "fb0509b3a73646c893e1e51a7a1b950f"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "33bb638585950823534db3a6d726b8bf"
server_key_iv: "e39359b793973375736626e95e9f786f"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.332173 18006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:46.354393 18006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:46.356711 18006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:46.356868 18006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:46.357365 18006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:46.357427 18006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.357471 18006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:46.357515 18006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:46.364555 18006 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:38301
I20260928 23:06:46.364636 18136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:38301 every 8 connection(s)
I20260928 23:06:46.364948 18006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:06:46.366859 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.357877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:51339 (local address 127.15.212.126:46345)
0928 23:06:46.357950 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.357953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.358146 (+ 193us) server_negotiation.cc:407] Connection header received
0928 23:06:46.358628 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.358629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.358643 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.358722 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.359544 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.360403 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.361525 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.361757 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.361847 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.364309 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.364311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.364311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.364320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.365789 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.366121 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.366121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.366122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.366172 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.366369 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.366370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.366370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.366464 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.366512 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.366735 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.366797 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 23:06:46.366997 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.357865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.65:51339)
0928 23:06:46.358018 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.358056 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.358526 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.358818 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.358826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.359171 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.359438 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.359443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.360530 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.360531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.361434 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.361436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.361874 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.361875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.362002 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.362469 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.362478 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.364194 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.365854 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.365858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.365859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.366055 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.366227 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.366228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.366228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.366315 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.366515 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.366518 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.366631 (+ 113us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.366709 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.366817 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":97,"threads_started":1}
I20260928 23:06:46.367482 18137 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46345
I20260928 23:06:46.367561 18137 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:46.367743 18137 heartbeater.cc:511] Master 127.15.212.126:46345 requested a full tablet report, sending...
I20260928 23:06:46.368257 17934 ts_manager.cc:194] Registered new tserver with Master: fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301)
I20260928 23:06:46.369190 17934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:51339
I20260928 23:06:46.370369 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18006
I20260928 23:06:46.370471 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:46.370652 16209 external_mini_cluster.cc:1428] Setting key 199149afafbf22097967998cfd0c9295
I20260928 23:06:46.374523 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.368567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:55809 (local address 127.15.212.126:46345)
0928 23:06:46.368605 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.368606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.368699 (+ 93us) server_negotiation.cc:407] Connection header received
0928 23:06:46.368761 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.368762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.368769 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.368811 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.369107 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.369699 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.370735 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.371006 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.371055 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.372086 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.372088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.372088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.372093 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.373498 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.373884 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.373884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.373885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.373929 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.374116 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.374117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.374117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.374209 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.374256 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.374420 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.374470 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:06:46.374694 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.368531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.65:55809)
0928 23:06:46.368641 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.368652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.368710 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.368816 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.368817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.368988 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.369049 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.369050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.369788 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.369789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.370626 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.370629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.371070 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.371071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.371150 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.371565 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.371570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.372003 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.373577 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.373578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.373578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.373814 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.373991 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.373992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.373992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.374059 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.374268 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.374269 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.374328 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.374404 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.374517 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:46.393456 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46345
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:46.472447 18146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:46.472628 18146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:46.472652 18146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:46.473713 18146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:46.473748 18146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:46.473768 18146 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 23:06:46.473779 18146 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:46.473796 18146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:46.475164 18146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:46345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:46.475328 18146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:46.475483 18146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:46.480046 18152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.480089 18154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.480257 18146 server_base.cc:1034] running on GCE node
W20260928 23:06:46.480094 18151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.480530 18146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:46.480722 18146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:46.481874 18146 hybrid_clock.cc:648] HybridClock initialized: now 1790636806481851 us; error 36 us; skew 500 ppm
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:46.484217 18146 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:46.484668 18146 webserver.cc:460] Webserver started at http://127.15.212.66:40781/ using document root <none> and password file <none>
I20260928 23:06:46.484853 18146 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:46.484890 18146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:46.484978 18146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:46.485872 18146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "7c85620fac63e6c3509a849b07fc4b14"
server_key_iv: "d69b1fbf3f351ebe4765b66c011fc0bd"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.486160 18146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "7c85620fac63e6c3509a849b07fc4b14"
server_key_iv: "d69b1fbf3f351ebe4765b66c011fc0bd"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.487354 18146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:46.487986 18161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.488142 18146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:46.488232 18146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "7c85620fac63e6c3509a849b07fc4b14"
server_key_iv: "d69b1fbf3f351ebe4765b66c011fc0bd"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.488310 18146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:46.505370 18146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:46.507541 18146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:46.507701 18146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:46.508224 18146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:46.508280 18146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.508320 18146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:46.508344 18146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:46.515532 18146 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35387
I20260928 23:06:46.515854 18146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:46.515837 18276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35387 every 8 connection(s)
I20260928 23:06:46.517355 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18146
I20260928 23:06:46.517442 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:46.517657 16209 external_mini_cluster.cc:1428] Setting key 56af48258649cce97ab0aeb12dd6613e
I20260928 23:06:46.518289 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.508612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34993 (local address 127.15.212.126:46345)
0928 23:06:46.508683 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.508686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.508984 (+ 298us) server_negotiation.cc:407] Connection header received
0928 23:06:46.509656 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.509657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.509667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.509711 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.510547 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.511084 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.512113 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.512312 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.512387 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.515299 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.515301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.515301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.515308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.516825 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.517203 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.517204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.517204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.517264 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.517569 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.517570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.517571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.517710 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.517780 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.518102 (+ 322us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.518187 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 23:06:46.518416 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.508620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.66:34993)
0928 23:06:46.508828 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.508876 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.509551 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.509789 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.509794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.510170 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.510459 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.510466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.511205 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.511206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.512023 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.512024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.512404 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.512404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.512552 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.513029 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.513040 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.515189 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.516894 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.516898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.516900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.517117 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.517334 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.517335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.517335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.517462 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.517810 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.517816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.517957 (+ 141us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.518056 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.518203 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":121,"threads_started":1}
I20260928 23:06:46.519894 18277 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46345
I20260928 23:06:46.519974 18277 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:46.520129 18277 heartbeater.cc:511] Master 127.15.212.126:46345 requested a full tablet report, sending...
I20260928 23:06:46.520452 17934 ts_manager.cc:194] Registered new tserver with Master: 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387)
I20260928 23:06:46.520895 17934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:34993
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:06:46.527161 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.521023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:42451 (local address 127.15.212.126:46345)
0928 23:06:46.521057 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.521059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.521070 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:06:46.521145 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.521146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.521156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.521206 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.521571 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.522344 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.523035 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.523377 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.523453 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.524309 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.524310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.524311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.524317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.526001 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.526328 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.526329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.526329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.526385 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.526626 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.526627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.526628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.526774 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.526851 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.527000 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.527080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:06:46.527170 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.520893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.66:42451)
0928 23:06:46.521019 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.521031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.521084 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.521209 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.521209 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.521397 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.521474 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.521483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.522441 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.522442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.522948 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.522950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.523458 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.523459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.523542 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.523971 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.523976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.524224 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.526099 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.526099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.526100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.526254 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.526453 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.526453 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.526454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.526549 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.526852 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.526853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.526902 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.526974 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.527071 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:46.540993 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46345
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:46.619386 18286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:46.619633 18286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:46.619683 18286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:46.620944 18286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:46.620999 18286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:46.621037 18286 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 23:06:46.621083 18286 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:06:46.621116 18286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:46.622555 18286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:46345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:46.622810 18286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:46.623040 18286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:46.627856 18294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.627920 18292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:46.627849 18291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:46.628156 18286 server_base.cc:1034] running on GCE node
I20260928 23:06:46.628364 18286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:46.628633 18286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:46.629791 18286 hybrid_clock.cc:648] HybridClock initialized: now 1790636806629778 us; error 29 us; skew 500 ppm
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:46.632220 18286 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:46.632642 18286 webserver.cc:460] Webserver started at http://127.15.212.67:45353/ using document root <none> and password file <none>
I20260928 23:06:46.632869 18286 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:46.632934 18286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:46.633085 18286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:46.634029 18286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "324c7f810636711098bafdc12a777384"
server_key_iv: "e9daa20ed2c60683ed508c018720ccc4"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.634403 18286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "324c7f810636711098bafdc12a777384"
server_key_iv: "e9daa20ed2c60683ed508c018720ccc4"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.635727 18286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:46.636428 18301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.636617 18286 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:46.636711 18286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
format_stamp: "Formatted at 2026-09-28 23:06:46 on dist-test-slave-x2t8"
server_key: "324c7f810636711098bafdc12a777384"
server_key_iv: "e9daa20ed2c60683ed508c018720ccc4"
server_key_version: "encryptionkey@0"
I20260928 23:06:46.636790 18286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:46.644798 18286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:46.646869 18286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:46.647008 18286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:46.647456 18286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:46.647516 18286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:46.647560 18286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:46.647584 18286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:46.654583 18286 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:39719
I20260928 23:06:46.654717 18416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:39719 every 8 connection(s)
I20260928 23:06:46.654997 18286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:46.657115 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.647932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:47571 (local address 127.15.212.126:46345)
0928 23:06:46.648002 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.648004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.648224 (+ 220us) server_negotiation.cc:407] Connection header received
0928 23:06:46.648721 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.648722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.648737 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.648821 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.649521 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.650290 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.651921 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.652177 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.652267 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.654711 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.654712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.654713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.654720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.656017 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.656359 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.656360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.656360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.656406 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.656676 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.656677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.656677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.656769 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.656819 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.657008 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.657059 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:06:46.657279 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.647863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.67:47571)
0928 23:06:46.648104 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.648137 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.648625 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.648891 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.648895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.649151 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.649448 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.649453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.650402 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.650403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.651826 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.651829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.652286 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.652287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.652472 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.652906 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.652913 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.654633 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.656091 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.656096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.656097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.656294 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.656490 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.656490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.656491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.656619 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.656824 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.656828 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.656924 (+ 96us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.656998 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.657103 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":75,"threads_started":1}
I20260928 23:06:46.657748 18417 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46345
I20260928 23:06:46.657827 18417 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:46.657991 18417 heartbeater.cc:511] Master 127.15.212.126:46345 requested a full tablet report, sending...
I20260928 23:06:46.658356 17934 ts_manager.cc:194] Registered new tserver with Master: 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:46.658826 17934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:47571
I20260928 23:06:46.663561 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.658777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:37771 (local address 127.15.212.126:46345)
0928 23:06:46.658813 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.658815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.658873 (+ 58us) server_negotiation.cc:407] Connection header received
0928 23:06:46.658942 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.658943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.658952 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.659006 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.659341 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.659762 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.660424 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.660679 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.660734 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.661446 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.661447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.661447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.661453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.662789 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.663056 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.663056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.663057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.663095 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.663227 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.663227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.663227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.663303 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.663348 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.663448 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.663509 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:06:46.663570 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.658703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.15.212.67:37771)
0928 23:06:46.658820 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.658831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.658888 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.659011 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.659011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.659208 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:46.659268 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.659269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.659872 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.659873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.660351 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.660352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.660742 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.660743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.660816 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.661130 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:46.661134 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:46.661375 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:46.662872 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.662873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.662873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.663011 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.663144 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:46.663145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:46.663145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:46.663190 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:46.663351 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:46.663352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:46.663383 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.663431 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.663510 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260928 23:06:46.664880 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18286
I20260928 23:06:46.664983 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:46.665179 16209 external_mini_cluster.cc:1428] Setting key 186655ab2c1c5b3ab290d7eb005d59ae
I20260928 23:06:46.665897 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 23:06:46 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:46.687291 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.681034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.15.212.126:46345)
0928 23:06:46.681086 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.681088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.681096 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:06:46.681128 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.681128 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.681137 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.681201 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:46.681740 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.682181 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.682883 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.683063 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.683160 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.685093 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:46.685095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:46.685095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:46.685104 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:46.686210 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.686558 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.686559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.686560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.686613 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:46.686815 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:46.686815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:46.686815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:46.686911 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:46.687019 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.687119 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.687186 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:06:46.688598 17934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:46.690035 17934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:46.696115 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.15.212.66:35387)
0928 23:06:46.692607 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.692611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.692620 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:46.692654 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.692655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.692669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.693146 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.693205 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.694211 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695252 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.695723 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695810 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.695866 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.695986 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.696020 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260928 23:06:46.696156 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35387 (local address 127.0.0.1:49508)
0928 23:06:46.692278 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.692315 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.692377 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.692932 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.692932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.692935 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.692977 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.692978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.694287 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.694288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.695162 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.695164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.695809 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.695890 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.695903 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.695969 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.696017 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":47,"threads_started":1}
I20260928 23:06:46.696424 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.15.212.67:39719)
0928 23:06:46.692588 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.692592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.692602 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:46.692649 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.692651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.692666 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.692756 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.692933 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.693662 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695093 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.695800 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.696215 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.696272 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.696297 (+ 25us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.696332 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":69,"threads_started":1}
I20260928 23:06:46.696431 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.15.212.65:38301)
0928 23:06:46.692338 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.692343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.692512 (+ 169us) server_negotiation.cc:407] Connection header received
0928 23:06:46.692656 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.692659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.692674 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.692783 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.692984 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.694070 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695268 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.695972 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.696031 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.696081 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.696302 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.696338 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260928 23:06:46.696174 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.0.0.1:53434)
0928 23:06:46.692278 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.692315 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.692378 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.692776 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.692779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.692782 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.692837 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.692839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.693749 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.693750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.694995 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.694998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.695902 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.695903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.695966 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.695970 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.696002 (+ 32us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.696057 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260928 23:06:46.696499 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.692076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38301 (local address 127.0.0.1:51116)
0928 23:06:46.692383 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.692394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.692554 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.692830 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.692831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.692833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.692897 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.692899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.694156 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.694157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.695186 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.695187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.696113 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.696114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.696184 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.696188 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.696233 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.696265 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260928 23:06:46.697346 18343 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 (DEFAULT_TABLE table=test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:46.697346 18211 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 (DEFAULT_TABLE table=test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:46.697397 18071 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 (DEFAULT_TABLE table=test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:46.697727 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad5f9022e934f3981594aef8e6d32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:46.697728 18343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad5f9022e934f3981594aef8e6d32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:46.697728 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad5f9022e934f3981594aef8e6d32d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:46.700560 18437 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Bootstrap starting.
I20260928 23:06:46.700923 18438 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Bootstrap starting.
I20260928 23:06:46.701192 18439 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Bootstrap starting.
I20260928 23:06:46.701381 18437 tablet_bootstrap.cc:654] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:46.701728 18437 log.cc:824] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:46.701776 18439 tablet_bootstrap.cc:654] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:46.701856 18438 tablet_bootstrap.cc:654] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:46.702090 18439 log.cc:824] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:46.702184 18438 log.cc:824] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:46.702466 18437 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: No bootstrap required, opened a new log
I20260928 23:06:46.702556 18437 ts_tablet_manager.cc:1397] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:46.702800 18439 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: No bootstrap required, opened a new log
I20260928 23:06:46.702880 18439 ts_tablet_manager.cc:1397] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:46.702910 18438 tablet_bootstrap.cc:492] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: No bootstrap required, opened a new log
I20260928 23:06:46.702977 18438 ts_tablet_manager.cc:1397] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:46.704308 18439 raft_consensus.cc:348] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.704505 18439 raft_consensus.cc:374] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:46.704562 18439 raft_consensus.cc:729] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Initialized, Role: FOLLOWER
I20260928 23:06:46.704562 18437 raft_consensus.cc:348] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.704687 18439 consensus_queue.cc:260] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.704733 18437 raft_consensus.cc:374] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:46.704768 18437 raft_consensus.cc:729] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Initialized, Role: FOLLOWER
I20260928 23:06:46.704882 18437 consensus_queue.cc:260] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.704950 18439 ts_tablet_manager.cc:1428] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:06:46.704937 18438 raft_consensus.cc:348] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.705080 18277 heartbeater.cc:503] Master 127.15.212.126:46345 was elected leader, sending a full tablet report...
I20260928 23:06:46.705082 18438 raft_consensus.cc:374] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:46.705117 18438 raft_consensus.cc:729] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Initialized, Role: FOLLOWER
I20260928 23:06:46.705135 18437 ts_tablet_manager.cc:1428] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:06:46.705262 18137 heartbeater.cc:503] Master 127.15.212.126:46345 was elected leader, sending a full tablet report...
I20260928 23:06:46.705257 18438 consensus_queue.cc:260] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.705507 18438 ts_tablet_manager.cc:1428] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:06:46.705611 18417 heartbeater.cc:503] Master 127.15.212.126:46345 was elected leader, sending a full tablet report...
W20260928 23:06:46.766518 18278 tablet.cc:2369] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:06:46.865664 18138 tablet.cc:2369] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:06:46.905606 18418 tablet.cc:2369] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:46.966385 18443 raft_consensus.cc:482] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:46.966509 18443 raft_consensus.cc:504] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.966732 18444 raft_consensus.cc:482] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:46.966815 18444 raft_consensus.cc:504] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.967113 18444 leader_election.cc:288] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719), 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387)
I20260928 23:06:46.967968 18443 leader_election.cc:288] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:46.970948 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38301 (local address 127.15.212.66:40427)
0928 23:06:46.967616 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.967626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.967688 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.967858 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.967859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.967860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.967931 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.967933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968950 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.968951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970027 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.970029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970718 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970792 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970795 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.970843 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.970879 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260928 23:06:46.970971 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35387 (local address 127.15.212.65:56843)
0928 23:06:46.967467 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.967478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.967561 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.967756 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.967756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.967759 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.967831 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.967833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968777 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.968778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.969996 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.969998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970731 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970796 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970799 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.970843 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.970878 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260928 23:06:46.971161 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:56843 (local address 127.15.212.66:35387)
0928 23:06:46.967470 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.967472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.967536 (+ 64us) server_negotiation.cc:407] Connection header received
0928 23:06:46.967642 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.967643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.967653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.967684 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.967909 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968691 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970094 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970612 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970682 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970687 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.971054 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.971096 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:06:46.971151 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:49347 (local address 127.15.212.67:39719)
0928 23:06:46.967576 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.967578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.967588 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:46.967663 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.967665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.967675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.967706 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.967909 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968687 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970239 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970754 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970793 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970799 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.971049 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.971084 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:06:46.971366 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.15.212.65:49347)
0928 23:06:46.967465 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.967478 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.967573 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.967781 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.967782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.967784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.967832 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.967833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968791 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.968792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970139 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.970141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970852 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970921 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970925 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.971236 (+ 311us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.971277 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":148,"thread_start_us":62,"threads_started":1}
I20260928 23:06:46.971580 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.15.212.66:41591)
0928 23:06:46.968072 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:46.968083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:46.968138 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:46.968372 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:46.968373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:46.968375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:46.968457 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.968458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.969319 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.969319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.970474 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:46.970476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.971263 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.971264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:46.971337 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.971341 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:46.971374 (+ 33us) client_negotiation.cc:240] Negotiation successful
0928 23:06:46.971414 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260928 23:06:46.971624 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.967638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:40427 (local address 127.15.212.65:38301)
0928 23:06:46.967673 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.967675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.967682 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:06:46.967758 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.967759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.967767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.967815 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.968190 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.968872 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970108 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.970613 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970679 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.970687 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.971542 (+ 855us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.971576 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:06:46.971637 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" is_pre_election: true
I20260928 23:06:46.971736 18229 raft_consensus.cc:2454] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:46.971848 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:46.968062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:41591 (local address 127.15.212.67:39719)
0928 23:06:46.968246 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:46.968248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:46.968254 (+ 6us) server_negotiation.cc:407] Connection header received
0928 23:06:46.968273 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:46.968273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:46.968289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:46.968513 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:46.968539 (+ 26us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.969201 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.970557 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:46.971149 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:46.971693 (+ 544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:46.971703 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:46.971726 (+ 23us) server_negotiation.cc:299] Negotiation successful
0928 23:06:46.971764 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260928 23:06:46.971901 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f" is_pre_election: true
I20260928 23:06:46.971928 18024 leader_election.cc:302] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9adef25c5fa14ec6a3f22f9de3b37cb8, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:46.972002 18091 raft_consensus.cc:2454] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 0.
I20260928 23:06:46.972199 18371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" is_pre_election: true
I20260928 23:06:46.972342 18371 raft_consensus.cc:2454] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:46.972049 18444 raft_consensus.cc:2781] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:46.972396 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" is_pre_election: true
I20260928 23:06:46.972461 18444 raft_consensus.cc:482] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:46.972496 18444 raft_consensus.cc:3037] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:46.972486 18370 raft_consensus.cc:2454] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 0.
I20260928 23:06:46.972618 18162 leader_election.cc:302] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9adef25c5fa14ec6a3f22f9de3b37cb8, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:46.972754 18443 raft_consensus.cc:2781] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:46.972811 18443 raft_consensus.cc:482] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:46.972842 18443 raft_consensus.cc:3037] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:46.973304 18444 raft_consensus.cc:504] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.973507 18444 leader_election.cc:288] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Requested vote from peers 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719), 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387)
I20260928 23:06:46.973552 18443 raft_consensus.cc:504] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.973605 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
I20260928 23:06:46.973605 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
I20260928 23:06:46.973680 18443 leader_election.cc:288] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 election: Requested vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:46.973693 18229 raft_consensus.cc:2379] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fb0509b3a73646c893e1e51a7a1b950f in current term 1: Already voted for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in this term.
I20260928 23:06:46.973678 18370 raft_consensus.cc:3037] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:46.973845 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f"
I20260928 23:06:46.973922 18091 raft_consensus.cc:2379] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in current term 1: Already voted for candidate fb0509b3a73646c893e1e51a7a1b950f in this term.
I20260928 23:06:46.973971 18371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad5f9022e934f3981594aef8e6d32d8" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
I20260928 23:06:46.974210 18165 leader_election.cc:302] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9adef25c5fa14ec6a3f22f9de3b37cb8; no voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f
I20260928 23:06:46.974339 18443 raft_consensus.cc:2726] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260928 23:06:46.974557 18370 raft_consensus.cc:2454] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 1.
I20260928 23:06:46.974742 18025 leader_election.cc:302] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f; no voters: 9adef25c5fa14ec6a3f22f9de3b37cb8
I20260928 23:06:46.974864 18444 raft_consensus.cc:2781] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:46.974964 18444 raft_consensus.cc:686] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Becoming Leader. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Running, Role: LEADER
I20260928 23:06:46.975066 18444 consensus_queue.cc:237] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } }
I20260928 23:06:46.975826 17933 catalog_manager.cc:5495] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65). New cstate: current_term: 1 leader_uuid: "fb0509b3a73646c893e1e51a7a1b950f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } health_report { overall_health: UNKNOWN } } }
Sep 28 23:06:47 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636806, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:47.072453 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.065097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46345 (local address 127.0.0.1:59600)
0928 23:06:47.065290 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.065302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.065394 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.065535 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.065536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.065858 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:47.065900 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.065901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.066571 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.066572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.067172 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.067174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.067530 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.067531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.067587 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.068182 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:47.068187 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:47.069676 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:47.071251 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:47.071258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:47.071259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:47.071514 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:47.071732 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:47.071732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:47.071733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:47.071833 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:47.072067 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:47.072073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:47.072114 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.072194 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.072304 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260928 23:06:47.072453 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.065177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.15.212.126:46345)
0928 23:06:47.065266 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.065269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.065377 (+ 108us) server_negotiation.cc:407] Connection header received
0928 23:06:47.065448 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.065449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.065459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.065531 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:47.065985 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.066482 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.067253 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.067430 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.067531 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.069760 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:47.069762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:47.069762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:47.069768 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:47.071153 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:47.071610 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:47.071611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:47.071611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:47.071671 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:47.071891 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:47.071892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:47.071892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:47.071986 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:47.072063 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.072228 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.072358 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":61}
I20260928 23:06:47.074385 17933 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:59600:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 23:06:47.074625 17933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:47.076159 18211 tablet_service.cc:1467] Processing CreateTablet for tablet 952dc9d7b6ed475aa1b833617f34e52c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b1888403c364ac1b6c5fc3902398483]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:47.076175 18343 tablet_service.cc:1467] Processing CreateTablet for tablet 952dc9d7b6ed475aa1b833617f34e52c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b1888403c364ac1b6c5fc3902398483]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:47.076172 18071 tablet_service.cc:1467] Processing CreateTablet for tablet 952dc9d7b6ed475aa1b833617f34e52c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b1888403c364ac1b6c5fc3902398483]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:06:47.076329 18343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952dc9d7b6ed475aa1b833617f34e52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.076329 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952dc9d7b6ed475aa1b833617f34e52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.076331 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 952dc9d7b6ed475aa1b833617f34e52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.077618 18439 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: Bootstrap starting.
I20260928 23:06:47.077818 18437 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: Bootstrap starting.
I20260928 23:06:47.078019 18439 tablet_bootstrap.cc:654] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.078531 18437 tablet_bootstrap.cc:654] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.078730 18439 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: No bootstrap required, opened a new log
I20260928 23:06:47.078786 18439 ts_tablet_manager.cc:1397] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:47.078913 18438 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: Bootstrap starting.
I20260928 23:06:47.078929 18439 raft_consensus.cc:348] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.078994 18439 raft_consensus.cc:374] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.079020 18439 raft_consensus.cc:729] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.079067 18439 consensus_queue.cc:260] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.079200 18439 ts_tablet_manager.cc:1428] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:47.079223 18443 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.079321 18443 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.079335 18438 tablet_bootstrap.cc:654] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.079839 18437 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: No bootstrap required, opened a new log
I20260928 23:06:47.079896 18437 ts_tablet_manager.cc:1397] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:47.080049 18437 raft_consensus.cc:348] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.080121 18437 raft_consensus.cc:374] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.080137 18437 raft_consensus.cc:729] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.080212 18437 consensus_queue.cc:260] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.080312 18437 ts_tablet_manager.cc:1428] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:47.080353 18444 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.080461 18444 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.080544 18438 tablet_bootstrap.cc:492] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: No bootstrap required, opened a new log
I20260928 23:06:47.080591 18438 ts_tablet_manager.cc:1397] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:47.080734 18438 raft_consensus.cc:348] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.080786 18438 raft_consensus.cc:374] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.080799 18438 raft_consensus.cc:729] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.080865 18438 consensus_queue.cc:260] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.080946 18438 ts_tablet_manager.cc:1428] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:47.081019 18445 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.081094 18445 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.143462 18445 raft_consensus.cc:482] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:47.143548 18445 raft_consensus.cc:504] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.143854 18445 leader_election.cc:288] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387)
I20260928 23:06:47.147670 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.144080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:46543 (local address 127.15.212.66:35387)
0928 23:06:47.144124 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.144126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.144279 (+ 153us) server_negotiation.cc:407] Connection header received
0928 23:06:47.144333 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.144334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.144341 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.144391 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:47.144553 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.145249 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.146777 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.147326 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.147400 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.147409 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.147579 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.147613 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:06:47.147670 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.144058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:56199 (local address 127.15.212.65:38301)
0928 23:06:47.144103 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.144105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.144116 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:06:47.144190 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.144191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.144202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.144277 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:47.144463 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.145245 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.146776 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.147329 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.147400 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.147409 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.147558 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.147597 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:06:47.147670 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.143993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38301 (local address 127.15.212.67:56199)
0928 23:06:47.144048 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.144061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.144108 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.144299 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.144300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.144302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:47.144373 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.144375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.145334 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.145335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.146689 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.146692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.147406 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.147407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.147483 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.147485 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.147542 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.147584 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":100}
I20260928 23:06:47.147679 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.143993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35387 (local address 127.15.212.67:46543)
0928 23:06:47.144194 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.144211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.144280 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.144409 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.144410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.144411 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:47.144493 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.144495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.145334 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.145335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.146689 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.146692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.147417 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.147418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.147479 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.147483 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.147570 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.147596 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":117,"thread_start_us":71,"threads_started":1}
I20260928 23:06:47.147961 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952dc9d7b6ed475aa1b833617f34e52c" candidate_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f" is_pre_election: true
I20260928 23:06:47.147961 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952dc9d7b6ed475aa1b833617f34e52c" candidate_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" is_pre_election: true
I20260928 23:06:47.148048 18091 raft_consensus.cc:2454] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2948ba4d62ca4737bf7eb5defbfbb410 in term 0.
I20260928 23:06:47.148057 18229 raft_consensus.cc:2454] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2948ba4d62ca4737bf7eb5defbfbb410 in term 0.
I20260928 23:06:47.148221 18304 leader_election.cc:302] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, 9adef25c5fa14ec6a3f22f9de3b37cb8; no voters:
I20260928 23:06:47.148391 18445 raft_consensus.cc:2781] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:47.148452 18445 raft_consensus.cc:482] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:47.148476 18445 raft_consensus.cc:3037] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.149103 18445 raft_consensus.cc:504] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.149246 18445 leader_election.cc:288] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [CANDIDATE]: Term 1 election: Requested vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387)
I20260928 23:06:47.149405 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952dc9d7b6ed475aa1b833617f34e52c" candidate_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f"
I20260928 23:06:47.149422 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "952dc9d7b6ed475aa1b833617f34e52c" candidate_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
I20260928 23:06:47.149492 18091 raft_consensus.cc:3037] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.149503 18229 raft_consensus.cc:3037] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.150112 18091 raft_consensus.cc:2454] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2948ba4d62ca4737bf7eb5defbfbb410 in term 1.
I20260928 23:06:47.150125 18229 raft_consensus.cc:2454] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2948ba4d62ca4737bf7eb5defbfbb410 in term 1.
I20260928 23:06:47.150266 18304 leader_election.cc:302] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, 9adef25c5fa14ec6a3f22f9de3b37cb8; no voters:
I20260928 23:06:47.150384 18445 raft_consensus.cc:2781] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:47.150511 18445 raft_consensus.cc:686] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 LEADER]: Becoming Leader. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Running, Role: LEADER
I20260928 23:06:47.150629 18445 consensus_queue.cc:237] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.151084 18462 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: TxnStatusTablet state changed. Reason: New leader 2948ba4d62ca4737bf7eb5defbfbb410. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.151185 18462 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:47.151362 18464 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 23:06:47.151362 17933 catalog_manager.cc:5495] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } health_report { overall_health: HEALTHY } } }
I20260928 23:06:47.151489 18464 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 23:06:47.172849 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.170242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.15.212.67:39719)
0928 23:06:47.170287 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.170289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.170298 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:47.170434 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.170435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.170452 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.170527 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:47.170672 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.171197 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.172044 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.172243 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.172324 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.172535 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.172630 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.172725 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.172764 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.172783 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:06:47.172868 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.170182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.0.0.1:53440)
0928 23:06:47.170250 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.170262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.170310 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.170543 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.170544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.170546 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:47.170596 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.170598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.171275 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.171276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.171972 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.171974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.172325 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.172326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.172387 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.172404 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.172702 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.172710 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.172758 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.172802 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:06:47.174520 18091 raft_consensus.cc:1261] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Refusing update from remote peer 2948ba4d62ca4737bf7eb5defbfbb410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.174515 18229 raft_consensus.cc:1261] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Refusing update from remote peer 2948ba4d62ca4737bf7eb5defbfbb410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.174803 18462 consensus_queue.cc:1035] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:47.174980 18462 consensus_queue.cc:1035] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:47.175935 18443 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: TxnStatusTablet state changed. Reason: New leader 2948ba4d62ca4737bf7eb5defbfbb410. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.176017 18443 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.176162 18444 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: TxnStatusTablet state changed. Reason: New leader 2948ba4d62ca4737bf7eb5defbfbb410. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.176241 18444 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.176707 18445 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.176790 18445 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:47.177211 18444 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.177294 18444 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P fb0509b3a73646c893e1e51a7a1b950f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.178002 18462 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.178072 18462 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 2948ba4d62ca4737bf7eb5defbfbb410: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:06:47.178859 18443 tablet_replica.cc:439] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } } }
I20260928 23:06:47.178944 18443 tablet_replica.cc:442] T 952dc9d7b6ed475aa1b833617f34e52c P 9adef25c5fa14ec6a3f22f9de3b37cb8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:06:47.181185 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.178703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.15.212.65:38301)
0928 23:06:47.178740 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.178742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.178749 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:06:47.178775 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.178776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.178795 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.178855 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:47.179042 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.179500 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.180360 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.180634 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.180732 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.180868 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.180962 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.181042 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.181093 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.181109 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:06:47.187114 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.183754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44869 (local address 127.15.212.67:39719)
0928 23:06:47.183805 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.183806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.184362 (+ 556us) server_negotiation.cc:407] Connection header received
0928 23:06:47.184424 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.184425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.184468 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.184524 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:47.184685 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.185423 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.186443 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.186643 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.186721 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.186899 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.186929 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.186981 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.187036 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.187054 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:06:47.187162 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.183658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.15.212.65:44869)
0928 23:06:47.184304 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.184316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.184375 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.184527 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.184528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.184529 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:47.184601 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.184603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.185522 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.185523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.186351 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.186353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.186724 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.186724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.186785 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.186805 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.186983 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.186984 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.187032 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.187069 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":79,"threads_started":1}
I20260928 23:06:47.189301 18370 raft_consensus.cc:1261] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.189292 18229 raft_consensus.cc:1261] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.189524 18450 consensus_queue.cc:1035] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:47.189693 18444 consensus_queue.cc:1035] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:47.190752 18469 mvcc.cc:205] Tried to move back new op lower bound from 7334448362245775360 to 7334448361370451968. Current Snapshot: MvccSnapshot[applied={T|T < 7334448362245775360}]
I20260928 23:06:47.191167 18468 mvcc.cc:205] Tried to move back new op lower bound from 7334448362245775360 to 7334448361370451968. Current Snapshot: MvccSnapshot[applied={T|T < 7334448362245775360}]
I20260928 23:06:47.191773 18467 mvcc.cc:205] Tried to move back new op lower bound from 7334448362245775360 to 7334448361370451968. Current Snapshot: MvccSnapshot[applied={T|T < 7334448362245775360}]
I20260928 23:06:47.198082 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.195157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:41879 (local address 127.15.212.65:38301)
0928 23:06:47.195187 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:47.195188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:47.195327 (+ 139us) server_negotiation.cc:407] Connection header received
0928 23:06:47.195414 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:47.195415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:47.195423 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:47.195471 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:47.195856 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.196279 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.197394 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.197621 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.197704 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.197873 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.197902 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.197943 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:47.198006 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 23:06:47.198023 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260928 23:06:47.198140 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:47.195118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38301 (local address 127.15.212.67:41879)
0928 23:06:47.195248 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:47.195266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:47.195338 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:47.195474 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:47.195474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:47.195476 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:47.195778 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.195780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.196365 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.196366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.197310 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:47.197312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:47.197706 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:47.197707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:47.197766 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:47.197781 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:47.197954 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:47.197956 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:06:47.198003 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:06:47.198043 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260928 23:06:47.206772 17933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:47.206861 17933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:47.207612 17933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Sending DeleteTablet for 3 replicas of tablet 1ad5f9022e934f3981594aef8e6d32d8
I20260928 23:06:47.207893 18343 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:47 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260928 23:06:47.207927 18071 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:47 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:51116
I20260928 23:06:47.208199 18488 tablet_replica.cc:330] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: stopping tablet replica
I20260928 23:06:47.208302 18488 raft_consensus.cc:2229] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:47.208335 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad5f9022e934f3981594aef8e6d32d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:47 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49508
I20260928 23:06:47.208374 18488 raft_consensus.cc:2258] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:47.208750 18488 ts_tablet_manager.cc:1905] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:47.208904 17933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:47.209026 17933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:47.210266 18343 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 (DEFAULT_TABLE table=test-table [id=f52cc4f32bfe4a7e8431432e5392991a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:47.210373 18343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad9d9208e3435ba10bb5d19850f301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.210378 18488 ts_tablet_manager.cc:1918] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:47.210435 18488 log.cc:1196] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/1ad5f9022e934f3981594aef8e6d32d8
I20260928 23:06:47.210664 18488 ts_tablet_manager.cc:1939] T 1ad5f9022e934f3981594aef8e6d32d8 P fb0509b3a73646c893e1e51a7a1b950f: Deleting consensus metadata
I20260928 23:06:47.211611 18438 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Bootstrap starting.
I20260928 23:06:47.211695 17918 catalog_manager.cc:4841] TS fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301): tablet 1ad5f9022e934f3981594aef8e6d32d8 (table test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]) successfully deleted
I20260928 23:06:47.212049 18071 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 (DEFAULT_TABLE table=test-table [id=f52cc4f32bfe4a7e8431432e5392991a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:47.212064 18438 tablet_bootstrap.cc:654] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.212154 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad9d9208e3435ba10bb5d19850f301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.213328 18437 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Bootstrap starting.
I20260928 23:06:47.213658 18211 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 (DEFAULT_TABLE table=test-table [id=f52cc4f32bfe4a7e8431432e5392991a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:47.213711 18437 tablet_bootstrap.cc:654] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.213781 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad9d9208e3435ba10bb5d19850f301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:47.214277 18437 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: No bootstrap required, opened a new log
I20260928 23:06:47.214313 18437 ts_tablet_manager.cc:1397] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:47.214437 18437 raft_consensus.cc:348] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.214489 18437 raft_consensus.cc:374] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.214507 18437 raft_consensus.cc:729] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.214545 18437 consensus_queue.cc:260] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.214618 18437 ts_tablet_manager.cc:1428] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:47.215029 18439 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Bootstrap starting.
I20260928 23:06:47.215397 18439 tablet_bootstrap.cc:654] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:47.216131 18439 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: No bootstrap required, opened a new log
I20260928 23:06:47.216189 18439 ts_tablet_manager.cc:1397] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:47.216322 18439 raft_consensus.cc:348] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.216377 18439 raft_consensus.cc:374] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.216398 18439 raft_consensus.cc:729] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.216457 18439 consensus_queue.cc:260] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.216534 18439 ts_tablet_manager.cc:1428] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:47.217520 18489 tablet_replica.cc:330] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: stopping tablet replica
I20260928 23:06:47.217658 18489 raft_consensus.cc:2229] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:47.217764 18489 raft_consensus.cc:2258] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:47.218156 18489 ts_tablet_manager.cc:1905] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:47.219914 18489 ts_tablet_manager.cc:1918] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:47.220019 18489 log.cc:1196] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/1ad5f9022e934f3981594aef8e6d32d8
I20260928 23:06:47.220314 18489 ts_tablet_manager.cc:1939] T 1ad5f9022e934f3981594aef8e6d32d8 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting consensus metadata
I20260928 23:06:47.220957 17920 catalog_manager.cc:4841] TS 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387): tablet 1ad5f9022e934f3981594aef8e6d32d8 (table test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]) successfully deleted
I20260928 23:06:47.217937 18487 tablet_replica.cc:330] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: stopping tablet replica
I20260928 23:06:47.220768 18438 tablet_bootstrap.cc:492] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: No bootstrap required, opened a new log
I20260928 23:06:47.221194 18438 ts_tablet_manager.cc:1397] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260928 23:06:47.221282 18487 raft_consensus.cc:2229] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:47.221326 18438 raft_consensus.cc:348] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.221402 18438 raft_consensus.cc:374] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:47.221427 18438 raft_consensus.cc:729] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Initialized, Role: FOLLOWER
I20260928 23:06:47.221365 18487 raft_consensus.cc:2258] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:47.221469 18438 consensus_queue.cc:260] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.221580 18438 ts_tablet_manager.cc:1428] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:47.221954 18487 ts_tablet_manager.cc:1905] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:47.223412 18487 ts_tablet_manager.cc:1918] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:47.223519 18487 log.cc:1196] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/1ad5f9022e934f3981594aef8e6d32d8
I20260928 23:06:47.223809 18487 ts_tablet_manager.cc:1939] T 1ad5f9022e934f3981594aef8e6d32d8 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting consensus metadata
I20260928 23:06:47.224200 17921 catalog_manager.cc:4841] TS 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719): tablet 1ad5f9022e934f3981594aef8e6d32d8 (table test-table [id=0e16d8802b5e4c73b76ff4979fba3b9a]) successfully deleted
I20260928 23:06:47.281304 18485 raft_consensus.cc:482] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:47.281401 18485 raft_consensus.cc:504] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.281544 18485 leader_election.cc:288] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:47.281749 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad9d9208e3435ba10bb5d19850f301" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" is_pre_election: true
I20260928 23:06:47.281751 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad9d9208e3435ba10bb5d19850f301" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" is_pre_election: true
I20260928 23:06:47.281827 18370 raft_consensus.cc:2454] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:47.281850 18229 raft_consensus.cc:2454] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:47.281950 18025 leader_election.cc:302] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:47.282018 18485 raft_consensus.cc:2781] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:47.282058 18485 raft_consensus.cc:482] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:47.282075 18485 raft_consensus.cc:3037] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.282970 18485 raft_consensus.cc:504] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.283110 18485 leader_election.cc:288] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Requested vote from peers 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:47.283272 18229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad9d9208e3435ba10bb5d19850f301" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
I20260928 23:06:47.283311 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad9d9208e3435ba10bb5d19850f301" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
I20260928 23:06:47.283354 18229 raft_consensus.cc:3037] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.283383 18370 raft_consensus.cc:3037] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:47.284078 18370 raft_consensus.cc:2454] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 1.
I20260928 23:06:47.284085 18229 raft_consensus.cc:2454] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 1.
I20260928 23:06:47.284224 18025 leader_election.cc:302] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:47.284300 18485 raft_consensus.cc:2781] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:47.284337 18485 raft_consensus.cc:686] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Becoming Leader. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Running, Role: LEADER
I20260928 23:06:47.284386 18485 consensus_queue.cc:237] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:47.284897 17934 catalog_manager.cc:5495] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65). New cstate: current_term: 1 leader_uuid: "fb0509b3a73646c893e1e51a7a1b950f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } health_report { overall_health: UNKNOWN } } }
I20260928 23:06:47.295421 18370 raft_consensus.cc:1261] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.295593 18485 consensus_queue.cc:1035] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:47.296051 18229 raft_consensus.cc:1261] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:47.296241 18485 consensus_queue.cc:1035] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 23:06:47.300292 18469 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 23:06:48.818030 17933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:48.818125 17933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:48.818851 17933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Sending DeleteTablet for 3 replicas of tablet d6ad9d9208e3435ba10bb5d19850f301
I20260928 23:06:48.819115 18071 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:48 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:51116
I20260928 23:06:48.819115 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:48 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49508
I20260928 23:06:48.819288 18343 tablet_service.cc:1514] Processing DeleteTablet for tablet d6ad9d9208e3435ba10bb5d19850f301 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:48 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260928 23:06:48.819473 18518 tablet_replica.cc:330] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: stopping tablet replica
I20260928 23:06:48.819478 18520 tablet_replica.cc:330] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: stopping tablet replica
I20260928 23:06:48.819553 18520 raft_consensus.cc:2229] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:48.819553 18518 raft_consensus.cc:2229] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:48.819597 18518 raft_consensus.cc:2258] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:48.819629 18520 raft_consensus.cc:2258] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:48.819797 18519 tablet_replica.cc:330] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: stopping tablet replica
I20260928 23:06:48.819852 18518 ts_tablet_manager.cc:1905] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:48.819926 18519 raft_consensus.cc:2229] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:48.819937 18520 ts_tablet_manager.cc:1905] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:48.820052 18519 raft_consensus.cc:2258] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:48.820322 18519 ts_tablet_manager.cc:1905] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:48.821870 18520 ts_tablet_manager.cc:1918] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 23:06:48.821918 18518 ts_tablet_manager.cc:1918] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 23:06:48.821973 18520 log.cc:1196] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/d6ad9d9208e3435ba10bb5d19850f301
I20260928 23:06:48.822049 18518 log.cc:1196] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/d6ad9d9208e3435ba10bb5d19850f301
I20260928 23:06:48.822237 18520 ts_tablet_manager.cc:1939] T d6ad9d9208e3435ba10bb5d19850f301 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting consensus metadata
I20260928 23:06:48.822373 18518 ts_tablet_manager.cc:1939] T d6ad9d9208e3435ba10bb5d19850f301 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting consensus metadata
I20260928 23:06:48.822479 18519 ts_tablet_manager.cc:1918] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 23:06:48.822561 18519 log.cc:1196] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/d6ad9d9208e3435ba10bb5d19850f301
I20260928 23:06:48.822553 17921 catalog_manager.cc:4841] TS 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719): tablet d6ad9d9208e3435ba10bb5d19850f301 (table test-table [id=f52cc4f32bfe4a7e8431432e5392991a]) successfully deleted
I20260928 23:06:48.822757 17920 catalog_manager.cc:4841] TS 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387): tablet d6ad9d9208e3435ba10bb5d19850f301 (table test-table [id=f52cc4f32bfe4a7e8431432e5392991a]) successfully deleted
I20260928 23:06:48.822858 18519 ts_tablet_manager.cc:1939] T d6ad9d9208e3435ba10bb5d19850f301 P fb0509b3a73646c893e1e51a7a1b950f: Deleting consensus metadata
I20260928 23:06:48.823246 17918 catalog_manager.cc:4841] TS fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301): tablet d6ad9d9208e3435ba10bb5d19850f301 (table test-table [id=f52cc4f32bfe4a7e8431432e5392991a]) successfully deleted
Sep 28 23:06:48 dist-test-slave-x2t8 krb5kdc[17889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636808, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 23:06:48 dist-test-slave-x2t8 krb5kdc[17889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636808, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:48.843152 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:48.835868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.15.212.126:46345)
0928 23:06:48.836005 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:48.836007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:48.836049 (+ 42us) server_negotiation.cc:407] Connection header received
0928 23:06:48.836133 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:48.836133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:48.836144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:48.836202 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:48.836803 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:48.837307 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:48.838034 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:48.838222 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:48.838284 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:48.840780 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:48.840781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:48.840781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:48.840787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:48.842098 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:48.842426 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:48.842427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:48.842427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:48.842474 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:48.842659 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:48.842659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:48.842659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:48.842753 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:48.842825 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:48.843002 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 23:06:48.843085 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":94,"threads_started":1}
I20260928 23:06:48.844759 17933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:48.844936 17933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:48.846184 18071 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (DEFAULT_TABLE table=test-table [id=5986e7754b844af8b63f696cd960b4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:48.846231 18343 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (DEFAULT_TABLE table=test-table [id=5986e7754b844af8b63f696cd960b4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:48.846212 18211 tablet_service.cc:1467] Processing CreateTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (DEFAULT_TABLE table=test-table [id=5986e7754b844af8b63f696cd960b4c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:48.846321 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:48.846339 18343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:48.846369 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:48.847703 18530 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Bootstrap starting.
I20260928 23:06:48.847827 18529 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Bootstrap starting.
I20260928 23:06:48.848081 18530 tablet_bootstrap.cc:654] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:48.848124 18529 tablet_bootstrap.cc:654] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:48.848419 18531 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Bootstrap starting.
I20260928 23:06:48.848663 18529 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: No bootstrap required, opened a new log
I20260928 23:06:48.848704 18529 ts_tablet_manager.cc:1397] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:48.848841 18531 tablet_bootstrap.cc:654] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:48.848842 18529 raft_consensus.cc:348] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.848903 18529 raft_consensus.cc:374] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:48.848948 18529 raft_consensus.cc:729] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Initialized, Role: FOLLOWER
I20260928 23:06:48.849006 18529 consensus_queue.cc:260] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.849191 18529 ts_tablet_manager.cc:1428] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:48.849210 18530 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: No bootstrap required, opened a new log
I20260928 23:06:48.849256 18530 ts_tablet_manager.cc:1397] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:48.849375 18530 raft_consensus.cc:348] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.849445 18530 raft_consensus.cc:374] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:48.849465 18530 raft_consensus.cc:729] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Initialized, Role: FOLLOWER
I20260928 23:06:48.849529 18530 consensus_queue.cc:260] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.849650 18530 ts_tablet_manager.cc:1428] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:48.850109 18531 tablet_bootstrap.cc:492] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: No bootstrap required, opened a new log
I20260928 23:06:48.850160 18531 ts_tablet_manager.cc:1397] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:06:48.850296 18531 raft_consensus.cc:348] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.850381 18531 raft_consensus.cc:374] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:48.850401 18531 raft_consensus.cc:729] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Initialized, Role: FOLLOWER
I20260928 23:06:48.850455 18531 consensus_queue.cc:260] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.850549 18531 ts_tablet_manager.cc:1428] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:48.965946 18510 raft_consensus.cc:482] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:48.966042 18510 raft_consensus.cc:504] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.966284 18510 leader_election.cc:288] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:48.966601 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd0d88d5e40ee9e4e4c7639b27a95" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" is_pre_election: true
I20260928 23:06:48.966579 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd0d88d5e40ee9e4e4c7639b27a95" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" is_pre_election: true
I20260928 23:06:48.966697 18231 raft_consensus.cc:2454] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:48.966697 18370 raft_consensus.cc:2454] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 0.
I20260928 23:06:48.966881 18025 leader_election.cc:302] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:48.967015 18510 raft_consensus.cc:2781] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:48.967062 18510 raft_consensus.cc:482] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:48.967077 18510 raft_consensus.cc:3037] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:48.967778 18510 raft_consensus.cc:504] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.967921 18510 leader_election.cc:288] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Requested vote from peers 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:48.968058 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd0d88d5e40ee9e4e4c7639b27a95" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
I20260928 23:06:48.968070 18231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27fbd0d88d5e40ee9e4e4c7639b27a95" candidate_uuid: "fb0509b3a73646c893e1e51a7a1b950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8"
I20260928 23:06:48.968134 18231 raft_consensus.cc:3037] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:48.968134 18370 raft_consensus.cc:3037] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:48.968933 18370 raft_consensus.cc:2454] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 1.
I20260928 23:06:48.968981 18231 raft_consensus.cc:2454] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb0509b3a73646c893e1e51a7a1b950f in term 1.
I20260928 23:06:48.969115 18025 leader_election.cc:302] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, fb0509b3a73646c893e1e51a7a1b950f; no voters:
I20260928 23:06:48.969220 18510 raft_consensus.cc:2781] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:48.969271 18510 raft_consensus.cc:686] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Becoming Leader. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Running, Role: LEADER
I20260928 23:06:48.969328 18510 consensus_queue.cc:237] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:48.969925 17933 catalog_manager.cc:5495] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65). New cstate: current_term: 1 leader_uuid: "fb0509b3a73646c893e1e51a7a1b950f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } health_report { overall_health: UNKNOWN } } }
I20260928 23:06:48.996416 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:48.994125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.15.212.65:38301)
0928 23:06:48.994247 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:48.994249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:48.994256 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:06:48.994276 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:48.994277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:48.994287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:48.994348 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:48.994502 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:48.994971 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:48.995723 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:48.995932 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:48.996015 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:48.996157 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:48.996191 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:48.996257 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:48.996310 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 23:06:48.996332 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
I20260928 23:06:48.998570 18370 raft_consensus.cc:1261] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:48.998706 18231 raft_consensus.cc:1261] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Refusing update from remote peer fb0509b3a73646c893e1e51a7a1b950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:48.998760 18497 consensus_queue.cc:1035] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:48.998888 18510 consensus_queue.cc:1035] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:49.008057 17934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:49.008172 17934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:49.009038 17934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Sending DeleteTablet for 3 replicas of tablet 27fbd0d88d5e40ee9e4e4c7639b27a95
I20260928 23:06:49.009312 18343 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:49 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260928 23:06:49.009330 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:49 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49508
I20260928 23:06:49.009336 18071 tablet_service.cc:1514] Processing DeleteTablet for tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:49 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:51116
I20260928 23:06:49.009410 18520 tablet_replica.cc:330] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: stopping tablet replica
I20260928 23:06:49.009416 18519 tablet_replica.cc:330] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: stopping tablet replica
I20260928 23:06:49.009466 18519 raft_consensus.cc:2229] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:49.009433 18518 tablet_replica.cc:330] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: stopping tablet replica
I20260928 23:06:49.009469 18520 raft_consensus.cc:2229] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:49.009518 18518 raft_consensus.cc:2229] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:49.009529 18519 raft_consensus.cc:2258] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:49.009549 18520 raft_consensus.cc:2258] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:49.009572 18518 raft_consensus.cc:2258] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:49.009938 18520 ts_tablet_manager.cc:1905] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:49.009953 18519 ts_tablet_manager.cc:1905] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:49.010047 18518 ts_tablet_manager.cc:1905] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:49.011344 18520 ts_tablet_manager.cc:1918] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:49.011384 18520 log.cc:1196] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/27fbd0d88d5e40ee9e4e4c7639b27a95
I20260928 23:06:49.011432 18518 ts_tablet_manager.cc:1918] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:49.011485 18518 log.cc:1196] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/27fbd0d88d5e40ee9e4e4c7639b27a95
I20260928 23:06:49.011540 18519 ts_tablet_manager.cc:1918] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:06:49.011579 18520 ts_tablet_manager.cc:1939] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting consensus metadata
I20260928 23:06:49.011585 18519 log.cc:1196] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/27fbd0d88d5e40ee9e4e4c7639b27a95
I20260928 23:06:49.011720 18518 ts_tablet_manager.cc:1939] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting consensus metadata
I20260928 23:06:49.011785 18519 ts_tablet_manager.cc:1939] T 27fbd0d88d5e40ee9e4e4c7639b27a95 P fb0509b3a73646c893e1e51a7a1b950f: Deleting consensus metadata
I20260928 23:06:49.011838 17921 catalog_manager.cc:4841] TS 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719): tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (table test-table [id=5986e7754b844af8b63f696cd960b4c2]) successfully deleted
I20260928 23:06:49.012050 17920 catalog_manager.cc:4841] TS 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387): tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (table test-table [id=5986e7754b844af8b63f696cd960b4c2]) successfully deleted
I20260928 23:06:49.012251 17918 catalog_manager.cc:4841] TS fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301): tablet 27fbd0d88d5e40ee9e4e4c7639b27a95 (table test-table [id=5986e7754b844af8b63f696cd960b4c2]) successfully deleted
I20260928 23:06:49.012224 17934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:49.012400 17934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:49.013672 18071 tablet_service.cc:1467] Processing CreateTablet for tablet b3c2323015ba4ee096d110636db24d10 (DEFAULT_TABLE table=test-table [id=f1b5d247d3de4854bd77cd3f34772dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:49.013688 18211 tablet_service.cc:1467] Processing CreateTablet for tablet b3c2323015ba4ee096d110636db24d10 (DEFAULT_TABLE table=test-table [id=f1b5d247d3de4854bd77cd3f34772dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:49.013794 18071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c2323015ba4ee096d110636db24d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:49.013808 18211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c2323015ba4ee096d110636db24d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:49.014587 18529 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Bootstrap starting.
I20260928 23:06:49.014616 18530 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f: Bootstrap starting.
I20260928 23:06:49.014716 18343 tablet_service.cc:1467] Processing CreateTablet for tablet b3c2323015ba4ee096d110636db24d10 (DEFAULT_TABLE table=test-table [id=f1b5d247d3de4854bd77cd3f34772dc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:49.014812 18343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3c2323015ba4ee096d110636db24d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:49.014971 18530 tablet_bootstrap.cc:654] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:49.014992 18529 tablet_bootstrap.cc:654] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:49.015573 18530 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f: No bootstrap required, opened a new log
I20260928 23:06:49.015657 18530 ts_tablet_manager.cc:1397] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:49.015776 18530 raft_consensus.cc:348] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.015828 18530 raft_consensus.cc:374] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:49.015841 18530 raft_consensus.cc:729] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0509b3a73646c893e1e51a7a1b950f, State: Initialized, Role: FOLLOWER
I20260928 23:06:49.015894 18530 consensus_queue.cc:260] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.015971 18531 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Bootstrap starting.
I20260928 23:06:49.015980 18530 ts_tablet_manager.cc:1428] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:49.016227 18529 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: No bootstrap required, opened a new log
I20260928 23:06:49.016288 18529 ts_tablet_manager.cc:1397] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:06:49.016302 18531 tablet_bootstrap.cc:654] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:49.016463 18529 raft_consensus.cc:348] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.016541 18529 raft_consensus.cc:374] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:49.016582 18529 raft_consensus.cc:729] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Initialized, Role: FOLLOWER
I20260928 23:06:49.016626 18529 consensus_queue.cc:260] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.016714 18529 ts_tablet_manager.cc:1428] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:49.016868 18531 tablet_bootstrap.cc:492] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: No bootstrap required, opened a new log
I20260928 23:06:49.016911 18531 ts_tablet_manager.cc:1397] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:49.017025 18531 raft_consensus.cc:348] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.017073 18531 raft_consensus.cc:374] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:49.017086 18531 raft_consensus.cc:729] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2948ba4d62ca4737bf7eb5defbfbb410, State: Initialized, Role: FOLLOWER
I20260928 23:06:49.017117 18531 consensus_queue.cc:260] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.017207 18531 ts_tablet_manager.cc:1428] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:49.305552 18534 raft_consensus.cc:482] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:49.305691 18534 raft_consensus.cc:504] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.305883 18534 leader_election.cc:288] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:49.306079 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c2323015ba4ee096d110636db24d10" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" is_pre_election: true
I20260928 23:06:49.306185 18370 raft_consensus.cc:2454] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 0.
I20260928 23:06:49.306130 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c2323015ba4ee096d110636db24d10" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f" is_pre_election: true
I20260928 23:06:49.306247 18091 raft_consensus.cc:2454] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 0.
I20260928 23:06:49.306336 18165 leader_election.cc:302] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, 9adef25c5fa14ec6a3f22f9de3b37cb8; no voters:
I20260928 23:06:49.306437 18534 raft_consensus.cc:2781] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:49.306480 18534 raft_consensus.cc:482] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:49.306495 18534 raft_consensus.cc:3037] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:49.307209 18534 raft_consensus.cc:504] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.307349 18534 leader_election.cc:288] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 election: Requested vote from peers fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301), 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719)
I20260928 23:06:49.307477 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c2323015ba4ee096d110636db24d10" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb0509b3a73646c893e1e51a7a1b950f"
I20260928 23:06:49.307514 18370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3c2323015ba4ee096d110636db24d10" candidate_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2948ba4d62ca4737bf7eb5defbfbb410"
I20260928 23:06:49.307579 18091 raft_consensus.cc:3037] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:49.307579 18370 raft_consensus.cc:3037] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:49.308143 18370 raft_consensus.cc:2454] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 1.
I20260928 23:06:49.308285 18091 raft_consensus.cc:2454] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9adef25c5fa14ec6a3f22f9de3b37cb8 in term 1.
I20260928 23:06:49.308317 18165 leader_election.cc:302] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2948ba4d62ca4737bf7eb5defbfbb410, 9adef25c5fa14ec6a3f22f9de3b37cb8; no voters:
I20260928 23:06:49.308420 18534 raft_consensus.cc:2781] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:49.308583 18534 raft_consensus.cc:686] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 LEADER]: Becoming Leader. State: Replica: 9adef25c5fa14ec6a3f22f9de3b37cb8, State: Running, Role: LEADER
I20260928 23:06:49.308673 18534 consensus_queue.cc:237] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } }
I20260928 23:06:49.309366 17934 catalog_manager.cc:5495] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66). New cstate: current_term: 1 leader_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9adef25c5fa14ec6a3f22f9de3b37cb8" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 } health_report { overall_health: UNKNOWN } } }
I20260928 23:06:49.372219 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:49.369655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.15.212.66:35387)
0928 23:06:49.369806 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:49.369807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:49.369814 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:06:49.369847 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:49.369848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:49.369866 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:49.369935 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:49.370033 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.370613 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.371338 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.371645 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.371705 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:49.371866 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:49.371982 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:49.372041 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:49.372083 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 23:06:49.372100 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260928 23:06:49.376951 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:49.374851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:47439 (local address 127.15.212.67:39719)
0928 23:06:49.374962 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:49.374968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:49.374976 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:06:49.375027 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:49.375028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:49.375036 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:49.375093 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:49.375187 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.375641 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.376357 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.376556 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.376624 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:49.376762 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:49.376786 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:49.376824 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:49.376879 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 23:06:49.376895 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260928 23:06:49.376976 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:49.374765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39719 (local address 127.15.212.66:47439)
0928 23:06:49.374928 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:49.374938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:49.374984 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:49.375103 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:49.375103 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:49.375105 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:49.375150 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:49.375151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.375706 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.375707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:49.376283 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:49.376285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.376628 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.376628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:49.376677 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:49.376688 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:49.376839 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:49.376840 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:06:49.376871 (+ 31us) client_negotiation.cc:240] Negotiation successful
0928 23:06:49.376907 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260928 23:06:49.378969 18091 raft_consensus.cc:1261] T b3c2323015ba4ee096d110636db24d10 P fb0509b3a73646c893e1e51a7a1b950f [term 1 FOLLOWER]: Refusing update from remote peer 9adef25c5fa14ec6a3f22f9de3b37cb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:49.379040 18370 raft_consensus.cc:1261] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Refusing update from remote peer 9adef25c5fa14ec6a3f22f9de3b37cb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:49.379201 18546 consensus_queue.cc:1035] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb0509b3a73646c893e1e51a7a1b950f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:49.379324 18534 consensus_queue.cc:1035] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2948ba4d62ca4737bf7eb5defbfbb410" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:49.386303 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:49.383918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57761 (local address 127.15.212.66:35387)
0928 23:06:49.383964 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:49.383966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:49.384076 (+ 110us) server_negotiation.cc:407] Connection header received
0928 23:06:49.384140 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:49.384141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:49.384151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:49.384204 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:49.384305 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.384815 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.385687 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.385889 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.385968 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:49.386102 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:49.386132 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:49.386177 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:49.386225 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 23:06:49.386243 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 23:06:49.386341 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:49.383867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35387 (local address 127.15.212.67:57761)
0928 23:06:49.383995 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:49.384004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:49.384082 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:49.384208 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:49.384209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:49.384211 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:06:49.384251 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:49.384252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.384889 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.384890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:49.385618 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:49.385620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:49.385969 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:49.385970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:49.386024 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:49.386028 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:06:49.386176 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:06:49.386178 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:06:49.386223 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 23:06:49.386262 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260928 23:06:49.386656 18501 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 23:06:50.909840 17933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:50.909935 17933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:50.910627 17933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2df0af5355b4b4dae3102f9dea6f234: Sending DeleteTablet for 3 replicas of tablet b3c2323015ba4ee096d110636db24d10
I20260928 23:06:50.910856 18211 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c2323015ba4ee096d110636db24d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:50 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49508
I20260928 23:06:50.910938 18071 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c2323015ba4ee096d110636db24d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:50 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:51116
I20260928 23:06:50.911011 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18006
I20260928 23:06:50.911222 18343 tablet_service.cc:1514] Processing DeleteTablet for tablet b3c2323015ba4ee096d110636db24d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:50 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260928 23:06:50.912343 18583 tablet_replica.cc:330] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: stopping tablet replica
I20260928 23:06:50.912420 18583 raft_consensus.cc:2229] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:50.912482 18583 raft_consensus.cc:2258] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:50.912601 18581 tablet_replica.cc:330] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: stopping tablet replica
I20260928 23:06:50.912703 18581 raft_consensus.cc:2229] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:50.912765 18583 ts_tablet_manager.cc:1905] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:50.912817 18581 raft_consensus.cc:2258] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:50.913046 18581 ts_tablet_manager.cc:1905] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:50.914426 18583 ts_tablet_manager.cc:1918] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 23:06:50.914486 18583 log.cc:1196] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/b3c2323015ba4ee096d110636db24d10
I20260928 23:06:50.914670 18581 ts_tablet_manager.cc:1918] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 23:06:50.914722 18583 ts_tablet_manager.cc:1939] T b3c2323015ba4ee096d110636db24d10 P 2948ba4d62ca4737bf7eb5defbfbb410: Deleting consensus metadata
I20260928 23:06:50.914743 18581 log.cc:1196] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/b3c2323015ba4ee096d110636db24d10
I20260928 23:06:50.915022 17921 catalog_manager.cc:4841] TS 2948ba4d62ca4737bf7eb5defbfbb410 (127.15.212.67:39719): tablet b3c2323015ba4ee096d110636db24d10 (table test-table [id=f1b5d247d3de4854bd77cd3f34772dc3]) successfully deleted
I20260928 23:06:50.915118 18581 ts_tablet_manager.cc:1939] T b3c2323015ba4ee096d110636db24d10 P 9adef25c5fa14ec6a3f22f9de3b37cb8: Deleting consensus metadata
I20260928 23:06:50.915421 17920 catalog_manager.cc:4841] TS 9adef25c5fa14ec6a3f22f9de3b37cb8 (127.15.212.66:35387): tablet b3c2323015ba4ee096d110636db24d10 (table test-table [id=f1b5d247d3de4854bd77cd3f34772dc3]) successfully deleted
W20260928 23:06:50.917668 17918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:38301 (error 108)
I20260928 23:06:50.917868 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18146
I20260928 23:06:50.920006 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:50.917992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38301 (local address 127.0.0.1:38990)
0928 23:06:50.919911 (+ 1919us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:50.919961 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:38301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1893,"thread_start_us":56,"threads_started":1}
W20260928 23:06:50.920141 17918 catalog_manager.cc:4584] TS fb0509b3a73646c893e1e51a7a1b950f (127.15.212.65:38301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3c2323015ba4ee096d110636db24d10: Network error: Client connection negotiation failed: client connection to 127.15.212.65:38301: connect: Connection refused (error 111)
I20260928 23:06:50.923571 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18286
I20260928 23:06:50.929025 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 17902
2026-09-28T23:06:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6016 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:50 dist-test-slave-x2t8 krb5kdc[18588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:50 dist-test-slave-x2t8 krb5kdc[18588](info): set up 2 sockets
Sep 28 23:06:50 dist-test-slave-x2t8 krb5kdc[18588](info): commencing operation
krb5kdc: starting...
W20260928 23:06:51.956223 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:52.067426 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39945
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:52.148684 18601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:52.148873 18601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:52.148895 18601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:52.150017 18601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:52.150059 18601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:52.150074 18601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:52.150086 18601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:52.150099 18601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:52.151795 18601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:52.152047 18601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:52.152264 18601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:52.157235 18609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:52.157250 18607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.157289 18601 server_base.cc:1034] running on GCE node
W20260928 23:06:52.157250 18606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.157688 18601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:52.157974 18601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:52.159112 18601 hybrid_clock.cc:648] HybridClock initialized: now 1790636812159092 us; error 36 us; skew 500 ppm
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:52.161461 18601 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:52.161924 18601 webserver.cc:460] Webserver started at http://127.15.212.126:46687/ using document root <none> and password file <none>
I20260928 23:06:52.162128 18601 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:52.162170 18601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:52.162313 18601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:52.163249 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "e3b2748f01e644f0a08073d1ae3ee0b8"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "92dcdb14221c7bc4d2dd62ad90736114"
server_key_iv: "5fe83f9efba4983382451202ca9501b9"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.163583 18601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "e3b2748f01e644f0a08073d1ae3ee0b8"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "92dcdb14221c7bc4d2dd62ad90736114"
server_key_iv: "5fe83f9efba4983382451202ca9501b9"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.164913 18601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:52.165591 18616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.165769 18601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:52.165854 18601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "e3b2748f01e644f0a08073d1ae3ee0b8"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "92dcdb14221c7bc4d2dd62ad90736114"
server_key_iv: "5fe83f9efba4983382451202ca9501b9"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.165922 18601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:52.183292 18601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:52.185072 18601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:52.185214 18601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:52.189016 18601 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:39945
I20260928 23:06:52.189050 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:39945 every 8 connection(s)
I20260928 23:06:52.189369 18601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:52.190153 18669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:52.191726 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18601
I20260928 23:06:52.191802 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:52.191977 16209 external_mini_cluster.cc:1428] Setting key b8f6f13e083651eef8f74887ba594b3e
I20260928 23:06:52.192638 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8: Bootstrap starting.
I20260928 23:06:52.193288 18669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:52.193590 18669 log.cc:824] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:52.194398 18669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8: No bootstrap required, opened a new log
I20260928 23:06:52.196637 18669 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } }
I20260928 23:06:52.196831 18669 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:52.196864 18669 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b2748f01e644f0a08073d1ae3ee0b8, State: Initialized, Role: FOLLOWER
I20260928 23:06:52.196980 18669 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } }
I20260928 23:06:52.197065 18669 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:52.197098 18669 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:52.197185 18669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:52.197726 18669 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } }
I20260928 23:06:52.197808 18669 leader_election.cc:302] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b2748f01e644f0a08073d1ae3ee0b8; no voters:
I20260928 23:06:52.197975 18669 leader_election.cc:288] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:52.198033 18674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:52.198233 18669 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:52.198248 18674 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [term 1 LEADER]: Becoming Leader. State: Replica: e3b2748f01e644f0a08073d1ae3ee0b8, State: Running, Role: LEADER
I20260928 23:06:52.198508 18674 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } }
I20260928 23:06:52.198875 18676 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } } }
I20260928 23:06:52.198863 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b2748f01e644f0a08073d1ae3ee0b8. Latest consensus state: current_term: 1 leader_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b2748f01e644f0a08073d1ae3ee0b8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39945 } } }
I20260928 23:06:52.198962 18676 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:52.198969 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8 [sys.catalog]: This master's current role is: LEADER
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:52.199414 18683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:52.199905 18683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:52.201923 18683 catalog_manager.cc:1347] Generated new cluster ID: 57305bd7f40944459819af1bfd473623
I20260928 23:06:52.201977 18683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:52.204070 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.192491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.15.212.126:39945)
0928 23:06:52.192671 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:52.192679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:52.192710 (+ 31us) server_negotiation.cc:407] Connection header received
0928 23:06:52.193263 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:52.193273 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:52.193332 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:52.193489 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:52.194191 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.195556 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.196326 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.196985 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.197090 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.199962 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:52.199966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:52.199967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:52.199977 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:52.202020 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.202421 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.202424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.202424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.202529 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.202826 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.202827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.202827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.203138 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:52.203289 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:52.203706 (+ 417us) server_negotiation.cc:299] Negotiation successful
0928 23:06:52.203841 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260928 23:06:52.210578 18683 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:52.211040 18683 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:52.219964 18683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b2748f01e644f0a08073d1ae3ee0b8: Generated new TSK 0
I20260928 23:06:52.220119 18683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:52.245728 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:52.324229 18695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:52.324448 18695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:52.324546 18695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:52.325727 18695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:52.325790 18695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:52.325834 18695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:52.327235 18695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:52.327549 18695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:52.327780 18695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:52.332661 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.332793 18695 server_base.cc:1034] running on GCE node
W20260928 23:06:52.332612 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:52.332616 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.333110 18695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:52.333352 18695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:52.334512 18695 hybrid_clock.cc:648] HybridClock initialized: now 1790636812334503 us; error 41 us; skew 500 ppm
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:52.336884 18695 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:52.337285 18695 webserver.cc:460] Webserver started at http://127.15.212.65:34443/ using document root <none> and password file <none>
I20260928 23:06:52.337546 18695 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:52.337615 18695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:52.337757 18695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:52.338770 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "4f56837870f449eeb63cde2918458cd4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "7142424653417ae85f96830a64dfd5df"
server_key_iv: "ca5fa705bf583430e1dfa4a8098cfc45"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.339120 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "4f56837870f449eeb63cde2918458cd4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "7142424653417ae85f96830a64dfd5df"
server_key_iv: "ca5fa705bf583430e1dfa4a8098cfc45"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.340384 18695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:06:52.341100 18710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.341262 18695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:52.341358 18695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "4f56837870f449eeb63cde2918458cd4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "7142424653417ae85f96830a64dfd5df"
server_key_iv: "ca5fa705bf583430e1dfa4a8098cfc45"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.341437 18695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:52.363772 18695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:52.365593 18695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:52.365756 18695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:52.365973 18695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:52.366279 18695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:52.366334 18695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.366385 18695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:52.366412 18695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.371986 18695 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:46813
I20260928 23:06:52.372069 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:46813 every 8 connection(s)
I20260928 23:06:52.372375 18695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:52.380252 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18695
I20260928 23:06:52.380342 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:52.380548 16209 external_mini_cluster.cc:1428] Setting key 5b68686c796b50c275bca9204ef5fff5
I20260928 23:06:52.381150 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.373113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:45529 (local address 127.15.212.126:39945)
0928 23:06:52.373181 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:52.373184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:52.373416 (+ 232us) server_negotiation.cc:407] Connection header received
0928 23:06:52.373859 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:52.373860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:52.373870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:52.373924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:52.374779 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.375260 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.376168 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.376355 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.376404 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.378345 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:52.378347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:52.378347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:52.378353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:52.379753 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.380105 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.380106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.380106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.380154 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.380426 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.380427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.380427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.380589 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:52.380674 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:52.380964 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 23:06:52.381052 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:06:52.381366 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.373118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39945 (local address 127.15.212.65:45529)
0928 23:06:52.373295 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:52.373330 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:52.373777 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:52.373972 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:52.373976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:52.374522 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:52.374702 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.374706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.375334 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.375334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.376096 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.376097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.376422 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.376423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.376536 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.376946 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:52.376953 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:52.378257 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:52.379821 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.379825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.379828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.380017 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.380219 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.380220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.380220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.380361 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.380679 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:52.380685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:52.380818 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 23:06:52.380939 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 23:06:52.381126 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260928 23:06:52.381824 18824 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39945
I20260928 23:06:52.381923 18824 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:52.382097 18824 heartbeater.cc:511] Master 127.15.212.126:39945 requested a full tablet report, sending...
I20260928 23:06:52.382579 18633 ts_manager.cc:194] Registered new tserver with Master: 4f56837870f449eeb63cde2918458cd4 (127.15.212.65:46813)
I20260928 23:06:52.383133 18633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:45529
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:52.404397 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:52.483017 18829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:52.483208 18829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:52.483229 18829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:52.484378 18829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:52.484421 18829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:52.484503 18829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:52.485948 18829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:52.486193 18829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:52.486415 18829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:52.491235 18834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:52.491255 18837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:52.491246 18835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.491307 18829 server_base.cc:1034] running on GCE node
I20260928 23:06:52.491621 18829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:52.491858 18829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:52.493018 18829 hybrid_clock.cc:648] HybridClock initialized: now 1790636812493007 us; error 28 us; skew 500 ppm
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:52.495312 18829 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:52.495668 18829 webserver.cc:460] Webserver started at http://127.15.212.66:36611/ using document root <none> and password file <none>
I20260928 23:06:52.495843 18829 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:52.495883 18829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:52.495967 18829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:52.496990 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "6e9b63e30873445394cde946734c60a4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "f30f6b2dc9cb76678a84804dc41ea01a"
server_key_iv: "7a2db00d8757bd557bcca2543a7eed51"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.497290 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "6e9b63e30873445394cde946734c60a4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "f30f6b2dc9cb76678a84804dc41ea01a"
server_key_iv: "7a2db00d8757bd557bcca2543a7eed51"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.498641 18829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:52.499387 18844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.499629 18829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:52.499699 18829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "6e9b63e30873445394cde946734c60a4"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "f30f6b2dc9cb76678a84804dc41ea01a"
server_key_iv: "7a2db00d8757bd557bcca2543a7eed51"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.499749 18829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:52.518173 18829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:52.519961 18829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:52.520061 18829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:52.520226 18829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:52.520558 18829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:52.520599 18829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.520620 18829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:52.520671 18829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.526048 18829 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35251
I20260928 23:06:52.526121 18957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35251 every 8 connection(s)
I20260928 23:06:52.526418 18829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:52.528571 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18829
I20260928 23:06:52.528650 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:52.528837 16209 external_mini_cluster.cc:1428] Setting key d9254107e3e15c4da0aeaa67ee348a30
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:52.535939 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.527105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:55641 (local address 127.15.212.126:39945)
0928 23:06:52.527180 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:52.527183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:52.527357 (+ 174us) server_negotiation.cc:407] Connection header received
0928 23:06:52.527877 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:52.527878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:52.527889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:52.527941 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:52.528961 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.529575 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.530538 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.530739 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.530824 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.533069 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:52.533070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:52.533070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:52.533077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:52.534550 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.535029 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.535029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.535030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.535076 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.535353 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.535354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.535354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.535447 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:52.535496 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:52.535819 (+ 323us) server_negotiation.cc:299] Negotiation successful
0928 23:06:52.535879 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:06:52.536223 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.527066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39945 (local address 127.15.212.66:55641)
0928 23:06:52.527233 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:52.527275 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:52.527770 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:52.528007 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:52.528011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:52.528647 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:52.528859 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.528882 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.529670 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.529671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.530447 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.530448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.530833 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.530834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.530963 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.531457 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:52.531464 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:52.532967 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:52.534632 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.534638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.534641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.534950 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.535142 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.535142 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.535143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.535286 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.535514 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:52.535520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:52.535679 (+ 159us) client_negotiation.cc:769] Sending connection context
0928 23:06:52.535805 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 23:06:52.535969 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260928 23:06:52.536696 18958 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39945
I20260928 23:06:52.536805 18958 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:52.536974 18958 heartbeater.cc:511] Master 127.15.212.126:39945 requested a full tablet report, sending...
I20260928 23:06:52.537371 18633 ts_manager.cc:194] Registered new tserver with Master: 6e9b63e30873445394cde946734c60a4 (127.15.212.66:35251)
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:06:52.537781 18633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:55641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:52.555753 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:52.637389 18963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:52.637673 18963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:52.637712 18963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:52.638891 18963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:52.638962 18963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:52.639003 18963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:52.640407 18963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:52.640697 18963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:52.640914 18963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:52.645920 18968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:52.645962 18971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.645990 18963 server_base.cc:1034] running on GCE node
W20260928 23:06:52.645993 18969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:52.646297 18963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:52.646534 18963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:52.647725 18963 hybrid_clock.cc:648] HybridClock initialized: now 1790636812647715 us; error 27 us; skew 500 ppm
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:52.650153 18963 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:52.650604 18963 webserver.cc:460] Webserver started at http://127.15.212.67:40515/ using document root <none> and password file <none>
I20260928 23:06:52.650832 18963 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:52.650899 18963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:52.651046 18963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:52.652135 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "83efbe57bf1c4480b06c1ff5d5850ddf"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "af770be03d908f5aaff1dc6a730e7c1a"
server_key_iv: "73697661d8494599d3b8ab2860f1ad6b"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.652585 18963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "83efbe57bf1c4480b06c1ff5d5850ddf"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "af770be03d908f5aaff1dc6a730e7c1a"
server_key_iv: "73697661d8494599d3b8ab2860f1ad6b"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.653839 18963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:52.654533 18978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.654700 18963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:52.654793 18963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "83efbe57bf1c4480b06c1ff5d5850ddf"
format_stamp: "Formatted at 2026-09-28 23:06:52 on dist-test-slave-x2t8"
server_key: "af770be03d908f5aaff1dc6a730e7c1a"
server_key_iv: "73697661d8494599d3b8ab2860f1ad6b"
server_key_version: "encryptionkey@0"
I20260928 23:06:52.654871 18963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:52.680397 18963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:52.682211 18963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:52.682369 18963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:52.682610 18963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:52.682931 18963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:52.682983 18963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.683022 18963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:52.683063 18963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:52.688951 18963 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:40691
I20260928 23:06:52.688990 19091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:40691 every 8 connection(s)
I20260928 23:06:52.689294 18963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:52.690207 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 18963
I20260928 23:06:52.690305 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:52.690488 16209 external_mini_cluster.cc:1428] Setting key 855d21ca17baa57085dbf64059245630
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[18588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636812, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:52.698668 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.689961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:44427 (local address 127.15.212.126:39945)
0928 23:06:52.690001 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:52.690003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:52.690205 (+ 202us) server_negotiation.cc:407] Connection header received
0928 23:06:52.690865 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:52.690866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:52.690877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:52.690920 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:52.691882 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.692372 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.693376 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.693558 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.693642 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.695880 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:52.695882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:52.695882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:52.695889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:52.697451 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.697839 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.697840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.697841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.697889 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:52.698099 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:52.698099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:52.698100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:52.698226 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:52.698291 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:52.698548 (+ 257us) server_negotiation.cc:299] Negotiation successful
0928 23:06:52.698605 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:06:52.698915 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:52.689913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39945 (local address 127.15.212.67:44427)
0928 23:06:52.690091 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:52.690121 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:52.690753 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:52.690997 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:52.691001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:52.691570 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:52.691762 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.691767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.692472 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.692473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.693273 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:52.693275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:52.693666 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:52.693667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:52.693791 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:52.694254 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:52.694262 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:52.695756 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:52.697554 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.697559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.697561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.697774 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.697948 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:52.697949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:52.697949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:52.698044 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:52.698288 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:52.698292 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:52.698411 (+ 119us) client_negotiation.cc:769] Sending connection context
0928 23:06:52.698519 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 23:06:52.698658 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"spinlock_wait_cycles":12160,"thread_start_us":80,"threads_started":1}
I20260928 23:06:52.699380 19092 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39945
I20260928 23:06:52.699481 19092 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:52.699646 19092 heartbeater.cc:511] Master 127.15.212.126:39945 requested a full tablet report, sending...
I20260928 23:06:52.699977 18633 ts_manager.cc:194] Registered new tserver with Master: 83efbe57bf1c4480b06c1ff5d5850ddf (127.15.212.67:40691)
I20260928 23:06:52.700387 18633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:44427
I20260928 23:06:52.701996 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 23:06:52.712339 18672 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:06:52.711426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.15.212.126:39945)
0928 23:06:52.711499 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:52.711501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:52.711619 (+ 118us) server_negotiation.cc:407] Connection header received
0928 23:06:52.711714 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:52.711715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:52.711726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:52.711779 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:52.712285 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43562: BlockingRecv error: recv got EOF from 127.0.0.1:43562 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:06:52.713380 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18695
I20260928 23:06:52.717864 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18829
I20260928 23:06:52.721983 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18963
I20260928 23:06:52.726167 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 18601
2026-09-28T23:06:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1795 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[19104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[19104](info): set up 2 sockets
Sep 28 23:06:52 dist-test-slave-x2t8 krb5kdc[19104](info): commencing operation
krb5kdc: starting...
W20260928 23:06:53.748230 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:53 dist-test-slave-x2t8 krb5kdc[19104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:53.862510 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42549
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:39469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:42549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:53.939575 19117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:53.939798 19117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:53.939832 19117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:53.941038 19117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:53.941109 19117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:53.941131 19117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:53.941152 19117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:53.941182 19117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:53.942605 19117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:42549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:53.942860 19117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:53.943064 19117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:53.947595 19125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:53.947597 19122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:53.947649 19123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:53.947830 19117 server_base.cc:1034] running on GCE node
I20260928 23:06:53.948073 19117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:53.948343 19117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:53.949504 19117 hybrid_clock.cc:648] HybridClock initialized: now 1790636813949495 us; error 37 us; skew 500 ppm
Sep 28 23:06:53 dist-test-slave-x2t8 krb5kdc[19104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636813, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:53.951808 19117 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:53.952208 19117 webserver.cc:460] Webserver started at http://127.15.212.126:38403/ using document root <none> and password file <none>
I20260928 23:06:53.952419 19117 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:53.952508 19117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:53.952669 19117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:53.953636 19117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "e7d015a23e1d43158b8800022eef98c2"
format_stamp: "Formatted at 2026-09-28 23:06:53 on dist-test-slave-x2t8"
server_key: "effeb47a0b594e1c98a41b445bd11cf4"
server_key_iv: "64999c97a0692771669f4b92a020a6e0"
server_key_version: "encryptionkey@0"
I20260928 23:06:53.953989 19117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "e7d015a23e1d43158b8800022eef98c2"
format_stamp: "Formatted at 2026-09-28 23:06:53 on dist-test-slave-x2t8"
server_key: "effeb47a0b594e1c98a41b445bd11cf4"
server_key_iv: "64999c97a0692771669f4b92a020a6e0"
server_key_version: "encryptionkey@0"
I20260928 23:06:53.955391 19117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:53.956120 19132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:53.956290 19117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:53.956385 19117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "e7d015a23e1d43158b8800022eef98c2"
format_stamp: "Formatted at 2026-09-28 23:06:53 on dist-test-slave-x2t8"
server_key: "effeb47a0b594e1c98a41b445bd11cf4"
server_key_iv: "64999c97a0692771669f4b92a020a6e0"
server_key_version: "encryptionkey@0"
I20260928 23:06:53.956496 19117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:53.966115 19117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:53.967792 19117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:53.967943 19117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:53.971555 19117 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:42549
I20260928 23:06:53.971607 19184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:42549 every 8 connection(s)
I20260928 23:06:53.972056 19117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:53.972743 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:53.974782 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: Bootstrap starting.
I20260928 23:06:53.975291 19185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:53.975524 19185 log.cc:824] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:53.976143 19185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: No bootstrap required, opened a new log
I20260928 23:06:53.976351 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19117
I20260928 23:06:53.976480 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:53.976660 16209 external_mini_cluster.cc:1428] Setting key c5d49e5021736436b28e316e71fb36de
I20260928 23:06:53.978816 19185 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } }
I20260928 23:06:53.978979 19185 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:53.979014 19185 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d015a23e1d43158b8800022eef98c2, State: Initialized, Role: FOLLOWER
I20260928 23:06:53.979169 19185 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } }
I20260928 23:06:53.979249 19185 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:53.979272 19185 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:53.979342 19185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:53.979938 19185 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } }
I20260928 23:06:53.980039 19185 leader_election.cc:302] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d015a23e1d43158b8800022eef98c2; no voters:
I20260928 23:06:53.980199 19185 leader_election.cc:288] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:53.980229 19190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:53.980424 19190 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [term 1 LEADER]: Becoming Leader. State: Replica: e7d015a23e1d43158b8800022eef98c2, State: Running, Role: LEADER
I20260928 23:06:53.980423 19185 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:53.980679 19190 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } }
I20260928 23:06:53.981805 19192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d015a23e1d43158b8800022eef98c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } } }
I20260928 23:06:53.981880 19192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:53.981946 19191 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d015a23e1d43158b8800022eef98c2. Latest consensus state: current_term: 1 leader_uuid: "e7d015a23e1d43158b8800022eef98c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d015a23e1d43158b8800022eef98c2" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42549 } } }
I20260928 23:06:53.982016 19191 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2 [sys.catalog]: This master's current role is: LEADER
W20260928 23:06:53.982401 19205 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:06:53.983314 19205 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:06:53.983421 19206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:53.983624 19206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:06:53 dist-test-slave-x2t8 krb5kdc[19104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:53.985261 19206 catalog_manager.cc:1347] Generated new cluster ID: 8ae05a4e21464001bd326ef58d2b2a4a
I20260928 23:06:53.985316 19206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:53.988674 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:53.977118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.15.212.126:42549)
0928 23:06:53.977338 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:53.977347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:53.977379 (+ 32us) server_negotiation.cc:407] Connection header received
0928 23:06:53.978783 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:53.978791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:53.978859 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:53.979005 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:53.979755 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:53.981169 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:53.982232 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:53.982638 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:53.982766 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:53.985466 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:53.985470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:53.985470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:53.985481 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:53.987097 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:53.987469 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:53.987472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:53.987472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:53.987575 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:53.987787 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:53.987788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:53.987789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:53.988019 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:53.988185 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:53.988280 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 23:06:53.988532 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":80,"threads_started":1}
I20260928 23:06:53.989455 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:06:53.992084 19206 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:53.992735 19206 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:53.997263 19206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d015a23e1d43158b8800022eef98c2: Generated new TSK 0
I20260928 23:06:53.997411 19206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:06:54.006644 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:54.003198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.15.212.126:42549)
0928 23:06:54.003238 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:54.003240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:54.003322 (+ 82us) server_negotiation.cc:407] Connection header received
0928 23:06:54.003402 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:54.003402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:54.003410 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:54.003456 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:54.003835 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:54.004300 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:54.005379 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:54.005578 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:54.005628 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:54.006252 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:54.006253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:54.006254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:06:54.006410 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:54.006424 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:06:54.006429 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:54.006475 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:54.006568 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 23:06:54.006588 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:06:54.008888 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19117
2026-09-28T23:06:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1280 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:54 dist-test-slave-x2t8 krb5kdc[19218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:54 dist-test-slave-x2t8 krb5kdc[19218](info): set up 2 sockets
Sep 28 23:06:54 dist-test-slave-x2t8 krb5kdc[19218](info): commencing operation
krb5kdc: starting...
W20260928 23:06:55.032326 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:55.149184 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:33923
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:33923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:55.231947 19231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:55.232172 19231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:55.232208 19231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:55.233494 19231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:55.233564 19231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:55.233587 19231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:55.233609 19231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:55.233637 19231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:55.235067 19231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:33923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:33923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:55.235333 19231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:55.235577 19231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:55.240141 19237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.240175 19239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.240146 19236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.240367 19231 server_base.cc:1034] running on GCE node
I20260928 23:06:55.240688 19231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:55.240975 19231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:55.242120 19231 hybrid_clock.cc:648] HybridClock initialized: now 1790636815242105 us; error 41 us; skew 500 ppm
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:55.250290 19231 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:55.250746 19231 webserver.cc:460] Webserver started at http://127.15.212.126:37177/ using document root <none> and password file <none>
I20260928 23:06:55.250981 19231 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:55.251045 19231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:55.251190 19231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:55.252239 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "55940e1802c849ed86a632f5bc2b7020"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "9aa239b3644f0983fe8a191c97a1038f"
server_key_iv: "f86c8de27e4e1576b988cc9d2c40fa04"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.252602 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "55940e1802c849ed86a632f5bc2b7020"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "9aa239b3644f0983fe8a191c97a1038f"
server_key_iv: "f86c8de27e4e1576b988cc9d2c40fa04"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.253983 19231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:55.254724 19246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.254894 19231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:55.254992 19231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "55940e1802c849ed86a632f5bc2b7020"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "9aa239b3644f0983fe8a191c97a1038f"
server_key_iv: "f86c8de27e4e1576b988cc9d2c40fa04"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.255074 19231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:55.297989 19231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:55.299934 19231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:55.300096 19231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:55.303947 19231 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:33923
I20260928 23:06:55.304013 19298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:33923 every 8 connection(s)
I20260928 23:06:55.304346 19231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:55.305390 19299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:55.307608 19299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020: Bootstrap starting.
I20260928 23:06:55.308199 19299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:55.308487 19299 log.cc:824] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:55.309165 19299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020: No bootstrap required, opened a new log
I20260928 23:06:55.310664 19299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } }
I20260928 23:06:55.310799 19299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:55.310855 19299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55940e1802c849ed86a632f5bc2b7020, State: Initialized, Role: FOLLOWER
I20260928 23:06:55.311009 19299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } }
I20260928 23:06:55.311079 19299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:55.311113 19299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:55.311162 19299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:55.311926 19299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } }
I20260928 23:06:55.312040 19299 leader_election.cc:302] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55940e1802c849ed86a632f5bc2b7020; no voters:
I20260928 23:06:55.312206 19299 leader_election.cc:288] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:55.312227 19302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:55.312364 19302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [term 1 LEADER]: Becoming Leader. State: Replica: 55940e1802c849ed86a632f5bc2b7020, State: Running, Role: LEADER
I20260928 23:06:55.312487 19299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:55.312511 19302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } }
I20260928 23:06:55.312903 19304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55940e1802c849ed86a632f5bc2b7020. Latest consensus state: current_term: 1 leader_uuid: "55940e1802c849ed86a632f5bc2b7020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } } }
I20260928 23:06:55.312903 19303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55940e1802c849ed86a632f5bc2b7020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55940e1802c849ed86a632f5bc2b7020" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33923 } } }
I20260928 23:06:55.313067 19304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:55.313074 19303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:55.313512 19311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:55.313916 19311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:55.314335 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19231
I20260928 23:06:55.314442 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:55.314600 16209 external_mini_cluster.cc:1428] Setting key b08813994e6523a9d4a03336bd8b29a5
I20260928 23:06:55.316056 19311 catalog_manager.cc:1347] Generated new cluster ID: 4d2a67ae9d6a4524a4e072b6a10034fe
I20260928 23:06:55.316114 19311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:55.326150 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.315034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35778 (local address 127.15.212.126:33923)
0928 23:06:55.315194 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:55.315203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:55.315304 (+ 101us) server_negotiation.cc:407] Connection header received
0928 23:06:55.315445 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:55.315452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:55.315506 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:55.315654 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:55.316525 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.317913 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.318997 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.319567 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.319666 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.322507 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:55.322511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:55.322511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:55.322526 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:55.324533 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.324916 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.324919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.324920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.325028 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.325215 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.325215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.325216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.325519 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:55.325678 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:55.325779 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:06:55.325915 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260928 23:06:55.333519 19311 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:55.334200 19311 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:55.337092 19311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55940e1802c849ed86a632f5bc2b7020: Generated new TSK 0
I20260928 23:06:55.337250 19311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:55.363140 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:55.440376 19325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:55.440601 19325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:55.440635 19325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:55.441751 19325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:55.441812 19325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:55.441857 19325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:55.443250 19325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:55.443492 19325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:55.443715 19325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:55.448386 19330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.448375 19331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.448498 19325 server_base.cc:1034] running on GCE node
W20260928 23:06:55.448376 19333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.448844 19325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:55.449108 19325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:55.450264 19325 hybrid_clock.cc:648] HybridClock initialized: now 1790636815450253 us; error 28 us; skew 500 ppm
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:55.452656 19325 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:55.453042 19325 webserver.cc:460] Webserver started at http://127.15.212.65:34147/ using document root <none> and password file <none>
I20260928 23:06:55.453269 19325 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:55.453333 19325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:55.453477 19325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:55.454440 19325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "e57c11782e6e4c0185e98dc1d83c8072"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "aac3a9516e01bb88e0a936eb02750b12"
server_key_iv: "2964a9cc30aaec3d76e581128a12edbd"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.454782 19325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "e57c11782e6e4c0185e98dc1d83c8072"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "aac3a9516e01bb88e0a936eb02750b12"
server_key_iv: "2964a9cc30aaec3d76e581128a12edbd"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.456022 19325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:55.456732 19340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.456895 19325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:55.456983 19325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "e57c11782e6e4c0185e98dc1d83c8072"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "aac3a9516e01bb88e0a936eb02750b12"
server_key_iv: "2964a9cc30aaec3d76e581128a12edbd"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.457057 19325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:55.474244 19325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:55.475839 19325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:55.475988 19325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:55.476207 19325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:55.476565 19325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:55.476614 19325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.476647 19325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:55.476686 19325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.482038 19325 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:33079
I20260928 23:06:55.482106 19453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:33079 every 8 connection(s)
I20260928 23:06:55.482407 19325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:06:55.487282 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19325
I20260928 23:06:55.487385 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:55.487607 16209 external_mini_cluster.cc:1428] Setting key 80e9837b442b91a2ca831cc1285f2138
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:55.491341 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.483079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:33923 (local address 127.15.212.65:53145)
0928 23:06:55.483259 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:55.483292 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:55.483730 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:55.483922 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:55.483924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:55.484530 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:55.484724 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.484729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.485351 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.485352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.486128 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.486130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.486449 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.486450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.486542 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.486930 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:55.486937 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:55.488420 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:55.490006 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.490012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.490014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.490318 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.490540 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.490541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.490541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.490633 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.490899 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:55.490903 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:55.491002 (+ 99us) client_negotiation.cc:769] Sending connection context
0928 23:06:55.491079 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 23:06:55.491196 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":117,"threads_started":1}
I20260928 23:06:55.491353 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.483112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:53145 (local address 127.15.212.126:33923)
0928 23:06:55.483197 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:55.483200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:55.483373 (+ 173us) server_negotiation.cc:407] Connection header received
0928 23:06:55.483808 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:55.483809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:55.483821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:55.483871 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:55.484798 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.485281 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.486197 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.486384 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.486433 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.488530 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:55.488531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:55.488531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:55.488538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:55.489928 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.490413 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.490414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.490415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.490474 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.490693 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.490694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.490694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.490837 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:55.490905 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:55.491123 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 23:06:55.491216 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 23:06:55.491982 19454 heartbeater.cc:347] Connected to a master server at 127.15.212.126:33923
I20260928 23:06:55.492079 19454 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:55.492254 19454 heartbeater.cc:511] Master 127.15.212.126:33923 requested a full tablet report, sending...
I20260928 23:06:55.492821 19263 ts_manager.cc:194] Registered new tserver with Master: e57c11782e6e4c0185e98dc1d83c8072 (127.15.212.65:33079)
I20260928 23:06:55.493789 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:53145
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:55.510914 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:55.588688 19459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:55.588867 19459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:55.588889 19459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:55.590003 19459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:55.590044 19459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:55.590101 19459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:55.591481 19459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:55.591732 19459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:55.591944 19459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:55.596714 19465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.596721 19467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.596764 19459 server_base.cc:1034] running on GCE node
W20260928 23:06:55.596721 19464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.597167 19459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:55.597357 19459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:55.598512 19459 hybrid_clock.cc:648] HybridClock initialized: now 1790636815598499 us; error 38 us; skew 500 ppm
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:55.600826 19459 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:55.601269 19459 webserver.cc:460] Webserver started at http://127.15.212.66:37419/ using document root <none> and password file <none>
I20260928 23:06:55.601492 19459 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:55.601553 19459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:55.601686 19459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:55.602627 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "7a1b5be3d62f4a4dadb3aa5701c83caa"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "b0aee441806474e88932a5f64b3e338a"
server_key_iv: "fc2211c8f8c8e63fccf54e4d37ee1879"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.602959 19459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "7a1b5be3d62f4a4dadb3aa5701c83caa"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "b0aee441806474e88932a5f64b3e338a"
server_key_iv: "fc2211c8f8c8e63fccf54e4d37ee1879"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.604193 19459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:06:55.604944 19474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.605116 19459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.605207 19459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "7a1b5be3d62f4a4dadb3aa5701c83caa"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "b0aee441806474e88932a5f64b3e338a"
server_key_iv: "fc2211c8f8c8e63fccf54e4d37ee1879"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.605293 19459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:55.622488 19459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:55.624818 19459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:55.624969 19459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:55.625239 19459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:55.625623 19459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:55.625670 19459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.625726 19459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:55.625756 19459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.631260 19459 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:38859
I20260928 23:06:55.631309 19587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:38859 every 8 connection(s)
I20260928 23:06:55.631594 19459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:55.635223 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19459
I20260928 23:06:55.635322 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:55.635509 16209 external_mini_cluster.cc:1428] Setting key 9a84ce6baa4e5ec2a3188fdc611419a0
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:55.640519 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.632234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:38555 (local address 127.15.212.126:33923)
0928 23:06:55.632292 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:55.632294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:55.632488 (+ 194us) server_negotiation.cc:407] Connection header received
0928 23:06:55.632948 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:55.632949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:55.632957 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:55.633012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:55.633832 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.634304 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.635241 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.635553 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.635634 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.637743 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:55.637744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:55.637745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:55.637752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:55.639081 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.639571 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.639572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.639572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.639624 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.639896 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.639897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.639897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.639988 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:55.640036 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:55.640352 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 23:06:55.640410 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:06:55.640825 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.632214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:33923 (local address 127.15.212.66:38555)
0928 23:06:55.632363 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:55.632402 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:55.632874 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:55.633062 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:55.633065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:55.633580 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:55.633758 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.633762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.634371 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.634371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.635164 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.635166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.635638 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.635639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.635788 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.636263 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:55.636272 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:55.637652 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:55.639160 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.639166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.639168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.639491 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.639693 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.639694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.639695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.639829 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.640050 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:55.640056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:55.640213 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 23:06:55.640340 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 23:06:55.640539 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260928 23:06:55.641321 19588 heartbeater.cc:347] Connected to a master server at 127.15.212.126:33923
I20260928 23:06:55.641404 19588 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:55.641598 19588 heartbeater.cc:511] Master 127.15.212.126:33923 requested a full tablet report, sending...
I20260928 23:06:55.641924 19263 ts_manager.cc:194] Registered new tserver with Master: 7a1b5be3d62f4a4dadb3aa5701c83caa (127.15.212.66:38859)
I20260928 23:06:55.642287 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:38555
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:55.659559 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:55.737426 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:55.737638 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:55.737661 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:55.738714 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:55.738746 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:55.738771 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:55.740104 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:33923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:55.740279 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:55.740432 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:55.745004 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.745039 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:55.745031 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:55.745182 19593 server_base.cc:1034] running on GCE node
I20260928 23:06:55.745415 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:55.745608 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:55.746740 19593 hybrid_clock.cc:648] HybridClock initialized: now 1790636815746728 us; error 27 us; skew 500 ppm
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:55.749055 19593 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:55.749470 19593 webserver.cc:460] Webserver started at http://127.15.212.67:35417/ using document root <none> and password file <none>
I20260928 23:06:55.749650 19593 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:55.749689 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:55.749773 19593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:55.750715 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "0e55b48e7efc4502b9f19b1f61fbcf58"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "6c83d8a8e7beaef46ada0a5af6fa346b"
server_key_iv: "05957402b95718275e172baefb524141"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.750996 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "0e55b48e7efc4502b9f19b1f61fbcf58"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "6c83d8a8e7beaef46ada0a5af6fa346b"
server_key_iv: "05957402b95718275e172baefb524141"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.752202 19593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:55.753001 19608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.753185 19593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:06:55.753278 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "0e55b48e7efc4502b9f19b1f61fbcf58"
format_stamp: "Formatted at 2026-09-28 23:06:55 on dist-test-slave-x2t8"
server_key: "6c83d8a8e7beaef46ada0a5af6fa346b"
server_key_iv: "05957402b95718275e172baefb524141"
server_key_version: "encryptionkey@0"
I20260928 23:06:55.753351 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:55.772357 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:55.774607 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:55.774752 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:55.774967 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:55.775293 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:55.775346 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.775388 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:55.775414 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:55.781126 19593 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:35931
I20260928 23:06:55.781174 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:35931 every 8 connection(s)
I20260928 23:06:55.781486 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:06:55.783450 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19593
I20260928 23:06:55.783550 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:55.783754 16209 external_mini_cluster.cc:1428] Setting key 46a9f282cd9484de40f02070dcd01e41
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:55.790256 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.782168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:36649 (local address 127.15.212.126:33923)
0928 23:06:55.782228 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:55.782230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:55.782430 (+ 200us) server_negotiation.cc:407] Connection header received
0928 23:06:55.782879 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:55.782880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:55.782889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:55.782949 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:55.784130 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.784682 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.785593 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.785770 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.785816 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.787826 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:55.787827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:55.787828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:55.787835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:55.789064 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.789400 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.789401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.789401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.789448 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.789694 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.789695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.789695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.789822 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:55.789891 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:55.790119 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 23:06:55.790188 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:06:55.790354 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.782166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:33923 (local address 127.15.212.67:36649)
0928 23:06:55.782311 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:55.782349 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:55.782803 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:55.783000 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:55.783002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:55.783715 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:55.783992 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.783999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.784748 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.784749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.785523 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:55.785525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.785816 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.785817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:55.785902 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.786389 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:55.786402 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:55.787720 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:55.789131 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.789135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.789137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.789339 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.789502 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:55.789503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:55.789503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:55.789610 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:55.789896 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:55.789899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:55.790000 (+ 101us) client_negotiation.cc:769] Sending connection context
0928 23:06:55.790074 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 23:06:55.790185 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260928 23:06:55.790838 19722 heartbeater.cc:347] Connected to a master server at 127.15.212.126:33923
I20260928 23:06:55.790911 19722 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:55.791101 19722 heartbeater.cc:511] Master 127.15.212.126:33923 requested a full tablet report, sending...
I20260928 23:06:55.791499 19263 ts_manager.cc:194] Registered new tserver with Master: 0e55b48e7efc4502b9f19b1f61fbcf58 (127.15.212.67:35931)
I20260928 23:06:55.791896 19263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:36649
I20260928 23:06:55.795351 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636815, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:55.816845 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:55.809961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.15.212.126:33923)
0928 23:06:55.810029 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:55.810031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:55.810151 (+ 120us) server_negotiation.cc:407] Connection header received
0928 23:06:55.810175 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:55.810176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:55.810185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:55.810233 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:55.810731 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.811205 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.811861 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:55.812054 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:55.812113 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:55.814524 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:55.814526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:55.814526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:55.814532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:55.815736 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.816025 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.816026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.816026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.816075 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:55.816233 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:55.816233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:55.816233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:55.816352 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:55.816423 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:55.816614 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 23:06:55.816730 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260928 23:06:55.817061 19263 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35780
I20260928 23:06:55.817600 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19325
I20260928 23:06:55.822885 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19459
I20260928 23:06:55.827281 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19593
I20260928 23:06:55.831446 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19231
2026-09-28T23:06:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1824 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19734](info): set up 2 sockets
Sep 28 23:06:55 dist-test-slave-x2t8 krb5kdc[19734](info): commencing operation
krb5kdc: starting...
W20260928 23:06:56.856222 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:56 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:56.970691 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37675
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:57.049532 19747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:57.049722 19747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:57.049744 19747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:57.050951 19747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:57.050993 19747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:57.051008 19747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:57.051059 19747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:57.051082 19747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:57.052582 19747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:57.052814 19747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:57.053031 19747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:57.057741 19752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:57.057832 19747 server_base.cc:1034] running on GCE node
W20260928 23:06:57.057761 19755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.057761 19753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:57.058207 19747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:57.058470 19747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:57.059638 19747 hybrid_clock.cc:648] HybridClock initialized: now 1790636817059628 us; error 34 us; skew 500 ppm
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:57.062089 19747 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:57.062448 19747 webserver.cc:460] Webserver started at http://127.15.212.126:40005/ using document root <none> and password file <none>
I20260928 23:06:57.062616 19747 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:57.062665 19747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:57.062747 19747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:57.063687 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "330ac2515d8443dd80c31ef2e3ff6865"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "7eedcdd9cf63465fd0e5d4711a36ed48"
server_key_iv: "ac6587e5814dc6206f3ce737d6e5e418"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.063997 19747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "330ac2515d8443dd80c31ef2e3ff6865"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "7eedcdd9cf63465fd0e5d4711a36ed48"
server_key_iv: "ac6587e5814dc6206f3ce737d6e5e418"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.065265 19747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:57.065896 19762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.066056 19747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:57.066141 19747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "330ac2515d8443dd80c31ef2e3ff6865"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "7eedcdd9cf63465fd0e5d4711a36ed48"
server_key_iv: "ac6587e5814dc6206f3ce737d6e5e418"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.066203 19747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:57.075284 19747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:57.077169 19747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:57.077425 19747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:57.081213 19747 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:37675
I20260928 23:06:57.081266 19814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:37675 every 8 connection(s)
I20260928 23:06:57.081733 19747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:57.082342 19815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:57.084427 19815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865: Bootstrap starting.
I20260928 23:06:57.084861 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19747
I20260928 23:06:57.084939 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:57.085062 19815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:57.085124 16209 external_mini_cluster.cc:1428] Setting key 54c7e7f3e5496c75facffe5b301cc762
I20260928 23:06:57.085356 19815 log.cc:824] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:57.086032 19815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865: No bootstrap required, opened a new log
I20260928 23:06:57.087791 19815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } }
I20260928 23:06:57.087935 19815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:57.087981 19815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330ac2515d8443dd80c31ef2e3ff6865, State: Initialized, Role: FOLLOWER
I20260928 23:06:57.088135 19815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } }
I20260928 23:06:57.088222 19815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:57.088265 19815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:57.088311 19815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:57.088865 19815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } }
I20260928 23:06:57.088977 19815 leader_election.cc:302] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 330ac2515d8443dd80c31ef2e3ff6865; no voters:
I20260928 23:06:57.089135 19815 leader_election.cc:288] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:57.089178 19820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:57.089370 19815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:57.089352 19820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [term 1 LEADER]: Becoming Leader. State: Replica: 330ac2515d8443dd80c31ef2e3ff6865, State: Running, Role: LEADER
I20260928 23:06:57.089524 19820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } }
I20260928 23:06:57.089834 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "330ac2515d8443dd80c31ef2e3ff6865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } } }
I20260928 23:06:57.089984 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:57.090158 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 330ac2515d8443dd80c31ef2e3ff6865. Latest consensus state: current_term: 1 leader_uuid: "330ac2515d8443dd80c31ef2e3ff6865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ac2515d8443dd80c31ef2e3ff6865" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37675 } } }
I20260928 23:06:57.090250 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:57.090428 19828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:57.090907 19828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:57.092587 19828 catalog_manager.cc:1347] Generated new cluster ID: 85addce85169497282d55d839d2091f8
I20260928 23:06:57.092646 19828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:57.096136 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.085549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.15.212.126:37675)
0928 23:06:57.085726 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.085736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.085786 (+ 50us) server_negotiation.cc:407] Connection header received
0928 23:06:57.086222 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.086229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.086275 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.086410 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.086999 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.087826 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.088731 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.089252 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.089368 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.092253 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.092256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.092256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.092267 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.094228 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.094622 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.094625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.094625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.094737 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.094942 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.094942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.094943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.095223 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.095369 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.095802 (+ 433us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.095929 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":61,"threads_started":1}
I20260928 23:06:57.102111 19828 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:57.102814 19828 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:57.113173 19828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 330ac2515d8443dd80c31ef2e3ff6865: Generated new TSK 0
I20260928 23:06:57.113353 19828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:57.145094 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:57.225270 19841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:57.225478 19841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:57.225517 19841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:57.226655 19841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:57.226715 19841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:57.226761 19841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:57.228216 19841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:57.228494 19841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:57.228721 19841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:57.233527 19846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.233529 19847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.233510 19849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:57.233785 19841 server_base.cc:1034] running on GCE node
I20260928 23:06:57.233986 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:57.234261 19841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:57.235430 19841 hybrid_clock.cc:648] HybridClock initialized: now 1790636817235423 us; error 34 us; skew 500 ppm
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:57.237921 19841 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:57.238390 19841 webserver.cc:460] Webserver started at http://127.15.212.65:37727/ using document root <none> and password file <none>
I20260928 23:06:57.238622 19841 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:57.238687 19841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:57.238826 19841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:57.239799 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "d30e4cb4c25b4c478d65ad3c5d535a87"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "ee2501c560ac9c3c8519a1825a3aeccf"
server_key_iv: "aae3664d18133bcad54b1200a9f1c313"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.240140 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "d30e4cb4c25b4c478d65ad3c5d535a87"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "ee2501c560ac9c3c8519a1825a3aeccf"
server_key_iv: "aae3664d18133bcad54b1200a9f1c313"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.241478 19841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:57.242249 19856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.242439 19841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:57.242534 19841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d30e4cb4c25b4c478d65ad3c5d535a87"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "ee2501c560ac9c3c8519a1825a3aeccf"
server_key_iv: "aae3664d18133bcad54b1200a9f1c313"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.242615 19841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:57.263517 19841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:57.265763 19841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:57.265913 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:57.266149 19841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:57.266443 19841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:57.266490 19841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.266538 19841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:57.266564 19841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.272220 19841 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:32829
I20260928 23:06:57.272320 19969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:32829 every 8 connection(s)
I20260928 23:06:57.272617 19841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:57.279721 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19841
I20260928 23:06:57.279829 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:57.280018 16209 external_mini_cluster.cc:1428] Setting key c40f2bef4a86b616af338ba87010c6e5
I20260928 23:06:57.281813 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.273232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37675 (local address 127.15.212.65:42307)
0928 23:06:57.273379 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:57.273418 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:57.274301 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:57.274577 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:57.274580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:57.274826 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:57.275013 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.275017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.275833 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.275834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.276699 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.276701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.277022 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.277023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.277140 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.277581 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:57.277589 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:57.278845 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:57.280416 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.280422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.280424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.280734 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.280955 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.280956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.280956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.281063 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.281350 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:57.281354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:57.281458 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 23:06:57.281550 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 23:06:57.281664 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260928 23:06:57.281811 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.273258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:42307 (local address 127.15.212.126:37675)
0928 23:06:57.273305 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.273307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.273488 (+ 181us) server_negotiation.cc:407] Connection header received
0928 23:06:57.274396 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.274397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.274407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.274467 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.275216 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.275756 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.276765 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.276959 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.277011 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.278922 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.278924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.278924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.278931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.280348 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.280816 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.280817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.280818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.280884 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.281128 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.281129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.281129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.281285 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.281355 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.281583 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.281670 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:06:57.282322 19970 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37675
I20260928 23:06:57.282423 19970 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:57.282603 19970 heartbeater.cc:511] Master 127.15.212.126:37675 requested a full tablet report, sending...
I20260928 23:06:57.283105 19779 ts_manager.cc:194] Registered new tserver with Master: d30e4cb4c25b4c478d65ad3c5d535a87 (127.15.212.65:32829)
I20260928 23:06:57.284026 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:42307
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:57.303730 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:57.381273 19975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:57.381474 19975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:57.381495 19975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:57.382642 19975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:57.382687 19975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:57.382714 19975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:57.384176 19975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:57.384402 19975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:57.384656 19975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:57.389358 19983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.389362 19980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.389362 19981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:57.389547 19975 server_base.cc:1034] running on GCE node
I20260928 23:06:57.389854 19975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:57.390081 19975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:57.391232 19975 hybrid_clock.cc:648] HybridClock initialized: now 1790636817391220 us; error 24 us; skew 500 ppm
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:57.393682 19975 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:57.394079 19975 webserver.cc:460] Webserver started at http://127.15.212.66:46109/ using document root <none> and password file <none>
I20260928 23:06:57.394289 19975 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:57.394351 19975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:57.394490 19975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:57.395488 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "725df73442fe44488202bb80dced9fd4"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "de107c5701b2aaf81a8bdf024ae7fa91"
server_key_iv: "a24ea9939a999d6fd4c8082633534cfb"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.395836 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "725df73442fe44488202bb80dced9fd4"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "de107c5701b2aaf81a8bdf024ae7fa91"
server_key_iv: "a24ea9939a999d6fd4c8082633534cfb"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.397215 19975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:06:57.397958 19990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.398118 19975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:57.398218 19975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "725df73442fe44488202bb80dced9fd4"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "de107c5701b2aaf81a8bdf024ae7fa91"
server_key_iv: "a24ea9939a999d6fd4c8082633534cfb"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.398286 19975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:57.407017 19975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:57.409181 19975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:57.409327 19975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:57.409536 19975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:57.409821 19975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:57.409869 19975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.409909 19975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:57.409937 19975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.415336 19975 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:39367
I20260928 23:06:57.415402 20103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:39367 every 8 connection(s)
I20260928 23:06:57.415701 19975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:57.417737 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 19975
I20260928 23:06:57.417852 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:57.418023 16209 external_mini_cluster.cc:1428] Setting key f43a567d2b9880d230a1f52860cdd0bb
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:57.425671 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.416391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:35743 (local address 127.15.212.126:37675)
0928 23:06:57.416477 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.416480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.416677 (+ 197us) server_negotiation.cc:407] Connection header received
0928 23:06:57.417541 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.417542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.417576 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.417647 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.418366 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.419041 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.420032 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.420342 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.420412 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.422708 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.422709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.422710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.422717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.424314 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.424740 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.424741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.424742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.424805 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.425031 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.425032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.425032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.425177 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.425244 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.425485 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.425575 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 23:06:57.425691 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.416381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37675 (local address 127.15.212.66:35743)
0928 23:06:57.416564 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:57.416597 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:57.417443 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:57.417708 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:57.417712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:57.418010 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:57.418283 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.418289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.419137 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.419137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.419966 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.419968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.420430 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.420431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.420582 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.421070 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:57.421079 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:57.422611 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:57.424392 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.424397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.424399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.424668 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.424873 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.424874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.424874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.424963 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.425244 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:57.425248 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:57.425352 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 23:06:57.425420 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 23:06:57.425529 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":116,"threads_started":1}
I20260928 23:06:57.426319 20104 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37675
I20260928 23:06:57.426419 20104 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:57.426595 20104 heartbeater.cc:511] Master 127.15.212.126:37675 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:06:57.426985 19779 ts_manager.cc:194] Registered new tserver with Master: 725df73442fe44488202bb80dced9fd4 (127.15.212.66:39367)
I20260928 23:06:57.427345 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:35743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:57.443378 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:57.521685 20109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:57.521869 20109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:57.521891 20109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:57.523031 20109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:57.523078 20109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:57.523115 20109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:57.524549 20109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:57.524716 20109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:57.524868 20109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:57.529428 20114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.529433 20117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:57.529414 20115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:57.529721 20109 server_base.cc:1034] running on GCE node
I20260928 23:06:57.529893 20109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:57.530063 20109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:57.531186 20109 hybrid_clock.cc:648] HybridClock initialized: now 1790636817531173 us; error 27 us; skew 500 ppm
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:57.533593 20109 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:57.533943 20109 webserver.cc:460] Webserver started at http://127.15.212.67:46693/ using document root <none> and password file <none>
I20260928 23:06:57.534112 20109 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:57.534152 20109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:57.534233 20109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:57.535133 20109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "c5e98fd38f2d44d5974000d475f69812"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "667eefc3bd8cfc6fabe383f31eab016d"
server_key_iv: "93845da9268595b41104908dfbc73446"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.535425 20109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "c5e98fd38f2d44d5974000d475f69812"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "667eefc3bd8cfc6fabe383f31eab016d"
server_key_iv: "93845da9268595b41104908dfbc73446"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.536724 20109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:57.537333 20124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.537497 20109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:57.537586 20109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "c5e98fd38f2d44d5974000d475f69812"
format_stamp: "Formatted at 2026-09-28 23:06:57 on dist-test-slave-x2t8"
server_key: "667eefc3bd8cfc6fabe383f31eab016d"
server_key_iv: "93845da9268595b41104908dfbc73446"
server_key_version: "encryptionkey@0"
I20260928 23:06:57.537668 20109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:57.559610 20109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:57.561784 20109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:57.561930 20109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:57.562175 20109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:57.562486 20109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:57.562520 20109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.562541 20109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:57.562580 20109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:57.568187 20109 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:42431
I20260928 23:06:57.568250 20237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:42431 every 8 connection(s)
I20260928 23:06:57.568624 20109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:57.577390 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.569413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:55823 (local address 127.15.212.126:37675)
0928 23:06:57.569485 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.569488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.569630 (+ 142us) server_negotiation.cc:407] Connection header received
0928 23:06:57.570611 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.570612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.570622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.570686 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.571219 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.571699 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.572595 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.572770 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.572823 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.574928 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.574929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.574929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.574935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.576163 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.576597 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.576598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.576599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.576668 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.576925 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.576925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.576925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.577021 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.577077 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.577272 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.577328 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:06:57.577564 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.569361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37675 (local address 127.15.212.67:55823)
0928 23:06:57.569506 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:57.569546 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:57.570502 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:57.570739 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:57.570742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:57.570959 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:57.571124 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.571127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.571774 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.571774 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.572521 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:57.572522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.572836 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.572837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:57.572927 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.573395 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:57.573404 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:57.574849 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:57.576233 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.576238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.576240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.576522 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.576762 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:57.576763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:57.576763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:57.576863 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:57.577081 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:57.577084 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:57.577185 (+ 101us) client_negotiation.cc:769] Sending connection context
0928 23:06:57.577261 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:06:57.577370 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260928 23:06:57.578043 20238 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37675
I20260928 23:06:57.578145 20238 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:57.578308 20238 heartbeater.cc:511] Master 127.15.212.126:37675 requested a full tablet report, sending...
I20260928 23:06:57.578531 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20109
I20260928 23:06:57.578650 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:57.578691 19779 ts_manager.cc:194] Registered new tserver with Master: c5e98fd38f2d44d5974000d475f69812 (127.15.212.67:42431)
I20260928 23:06:57.578840 16209 external_mini_cluster.cc:1428] Setting key 4c54c5e997a6d64581c9a9d934812b47
I20260928 23:06:57.579119 19779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:55823
I20260928 23:06:57.579656 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:06:57.601711 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.594418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.15.212.65:32829)
0928 23:06:57.594624 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.594629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.594639 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:57.594685 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.594687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.594703 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.594774 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.595375 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.596060 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.596807 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.597111 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.597171 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.599225 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.599231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.599231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.599236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.600637 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.600998 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.601001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.601001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.601054 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.601264 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.601264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.601265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.601400 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.601502 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.601572 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.601632 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[19734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:57.609077 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:57.602886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.15.212.126:37675)
0928 23:06:57.602940 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:57.602942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:57.602952 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:06:57.603021 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:57.603022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:57.603032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:57.603087 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:57.603543 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.604018 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.604760 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:57.604960 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:57.605038 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:57.606929 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:57.606930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:57.606931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:57.606937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:57.608172 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.608427 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.608428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.608428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.608539 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:57.608693 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:57.608693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:57.608693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:57.608816 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:57.608884 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:57.608964 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 23:06:57.609016 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260928 23:06:57.609300 19779 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342
I20260928 23:06:57.609604 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19841
I20260928 23:06:57.614249 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19975
I20260928 23:06:57.618387 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20109
I20260928 23:06:57.622429 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 19747
2026-09-28T23:06:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1790 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[20250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[20250](info): set up 2 sockets
Sep 28 23:06:57 dist-test-slave-x2t8 krb5kdc[20250](info): commencing operation
krb5kdc: starting...
W20260928 23:06:58.652241 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:06:58 dist-test-slave-x2t8 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:06:58.763145 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37653
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:58.840858 20263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:58.841068 20263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:58.841104 20263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:58.842208 20263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:58.842269 20263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:58.842293 20263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:58.842320 20263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:58.842347 20263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:58.843775 20263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:58.844026 20263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:58.844244 20263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:58.848826 20271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:58.848872 20268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:58.848918 20263 server_base.cc:1034] running on GCE node
W20260928 23:06:58.848865 20269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:58.849279 20263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:58.849581 20263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:58.850744 20263 hybrid_clock.cc:648] HybridClock initialized: now 1790636818850722 us; error 35 us; skew 500 ppm
Sep 28 23:06:58 dist-test-slave-x2t8 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636818, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:58.853017 20263 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:06:58.853354 20263 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:58.853402 20263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:58.853485 20263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:58.854408 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "74146704c52747819abd66122c0bf5d8"
format_stamp: "Formatted at 2026-09-28 23:06:58 on dist-test-slave-x2t8"
server_key: "f44767655713f3b224aeef5b86d58798"
server_key_iv: "470765e46fd38e4a3697e6cc6d19cf8e"
server_key_version: "encryptionkey@0"
I20260928 23:06:58.854682 20263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "74146704c52747819abd66122c0bf5d8"
format_stamp: "Formatted at 2026-09-28 23:06:58 on dist-test-slave-x2t8"
server_key: "f44767655713f3b224aeef5b86d58798"
server_key_iv: "470765e46fd38e4a3697e6cc6d19cf8e"
server_key_version: "encryptionkey@0"
I20260928 23:06:58.855819 20263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:58.856403 20277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:58.856600 20263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:58.856691 20263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "74146704c52747819abd66122c0bf5d8"
format_stamp: "Formatted at 2026-09-28 23:06:58 on dist-test-slave-x2t8"
server_key: "f44767655713f3b224aeef5b86d58798"
server_key_iv: "470765e46fd38e4a3697e6cc6d19cf8e"
server_key_version: "encryptionkey@0"
I20260928 23:06:58.856757 20263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:58.872804 20263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:58.874537 20263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:58.874681 20263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:58.878364 20263 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:37653
I20260928 23:06:58.878389 20329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:37653 every 8 connection(s)
I20260928 23:06:58.878659 20263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:58.879495 20330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:58.881600 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: Bootstrap starting.
I20260928 23:06:58.882179 20330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:58.882476 20330 log.cc:824] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:58.883116 20330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: No bootstrap required, opened a new log
I20260928 23:06:58.884644 20330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } }
I20260928 23:06:58.884766 20330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:58.884788 20330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74146704c52747819abd66122c0bf5d8, State: Initialized, Role: FOLLOWER
I20260928 23:06:58.884954 20330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } }
I20260928 23:06:58.885013 20330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:58.885035 20330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:58.885061 20330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:58.885828 20330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } }
I20260928 23:06:58.885911 20330 leader_election.cc:302] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74146704c52747819abd66122c0bf5d8; no voters:
I20260928 23:06:58.886076 20330 leader_election.cc:288] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:58.886145 20333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:58.886327 20330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:58.886358 20333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [term 1 LEADER]: Becoming Leader. State: Replica: 74146704c52747819abd66122c0bf5d8, State: Running, Role: LEADER
I20260928 23:06:58.886482 20333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } }
I20260928 23:06:58.886902 20334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74146704c52747819abd66122c0bf5d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } } }
I20260928 23:06:58.886992 20334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:58.887195 20342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:58.887226 20335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74146704c52747819abd66122c0bf5d8. Latest consensus state: current_term: 1 leader_uuid: "74146704c52747819abd66122c0bf5d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74146704c52747819abd66122c0bf5d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37653 } } }
I20260928 23:06:58.887444 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20263
I20260928 23:06:58.887538 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:58.887557 20335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8 [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:58.887728 16209 external_mini_cluster.cc:1428] Setting key de6d4d4f7d39d9980e84c571acffadb2
I20260928 23:06:58.887742 20342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:58.889259 20342 catalog_manager.cc:1347] Generated new cluster ID: d3f3c35195c848c29e2c7f45731dfd70
I20260928 23:06:58.889310 20342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:06:58 dist-test-slave-x2t8 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:58.898999 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:58.888321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.15.212.126:37653)
0928 23:06:58.888496 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:58.888505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:58.888547 (+ 42us) server_negotiation.cc:407] Connection header received
0928 23:06:58.888652 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:58.888657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:58.888698 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:58.888848 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:58.889687 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:58.891025 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:58.892071 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:58.892708 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:58.892800 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:58.895442 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:58.895446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:58.895446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:58.895457 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:58.897385 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:58.897800 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:58.897802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:58.897803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:58.897909 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:58.898104 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:58.898104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:58.898105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:58.898409 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:58.898551 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:58.898659 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 23:06:58.898793 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260928 23:06:58.910313 20342 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:58.911007 20342 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:58.917514 20342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: Generated new TSK 0
I20260928 23:06:58.917672 20342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:06:58.942801 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:59.021210 20356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:59.021404 20356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:59.021427 20356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:59.022528 20356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:59.022575 20356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:59.022601 20356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:59.024045 20356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:59.024224 20356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:59.024380 20356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:59.029206 20362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.029206 20364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.029245 20361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.029263 20356 server_base.cc:1034] running on GCE node
I20260928 23:06:59.029659 20356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:59.029913 20356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:59.031064 20356 hybrid_clock.cc:648] HybridClock initialized: now 1790636819031050 us; error 29 us; skew 500 ppm
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:59.033586 20356 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:06:59.033862 20356 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:59.033905 20356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:59.033988 20356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:59.034916 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "0bb3fa3d606b4532b54f52744ffcaff5"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "c922aed4e0276c7e32bea094041f860c"
server_key_iv: "b2abd60fc30c631a16ee2164fe985217"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.035211 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "0bb3fa3d606b4532b54f52744ffcaff5"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "c922aed4e0276c7e32bea094041f860c"
server_key_iv: "b2abd60fc30c631a16ee2164fe985217"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.036326 20356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:06:59.037022 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.037169 20356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.037264 20356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "0bb3fa3d606b4532b54f52744ffcaff5"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "c922aed4e0276c7e32bea094041f860c"
server_key_iv: "b2abd60fc30c631a16ee2164fe985217"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.037333 20356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:59.058285 20356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:59.060755 20356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:59.060909 20356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:59.061120 20356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:59.061416 20356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:59.061465 20356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.061498 20356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:59.061527 20356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.067001 20356 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:43039
I20260928 23:06:59.067049 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:43039 every 8 connection(s)
I20260928 23:06:59.067303 20356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:59.075820 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.067975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:38833 (local address 127.15.212.126:37653)
0928 23:06:59.068032 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.068034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.068222 (+ 188us) server_negotiation.cc:407] Connection header received
0928 23:06:59.068760 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.068761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.068772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.068826 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.069704 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.070202 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.071170 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.071368 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.071446 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.073344 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.073345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.073346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:59.073353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.074504 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.074892 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.074893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.074894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.074949 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.075187 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.075188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.075188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.075308 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.075377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.075614 (+ 237us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.075732 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 23:06:59.075959 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.067939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37653 (local address 127.15.212.65:38833)
0928 23:06:59.068108 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.068148 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.068664 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.068900 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.068903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.069138 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:59.069606 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.069610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.070302 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.070303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.071080 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.071082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.071453 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.071454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.071567 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.071977 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:59.071984 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:59.073252 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:59.074595 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.074601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.074603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.074807 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.075011 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.075012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.075013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.075116 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.075369 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:59.075374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:59.075496 (+ 122us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.075593 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.075748 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260928 23:06:59.076364 20484 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37653
I20260928 23:06:59.076479 20484 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:59.076601 20484 heartbeater.cc:511] Master 127.15.212.126:37653 requested a full tablet report, sending...
I20260928 23:06:59.077005 20294 ts_manager.cc:194] Registered new tserver with Master: 0bb3fa3d606b4532b54f52744ffcaff5 (127.15.212.65:43039)
I20260928 23:06:59.077126 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20356
I20260928 23:06:59.077240 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:59.077414 16209 external_mini_cluster.cc:1428] Setting key e30884feca0d465418948abe2e35ac26
I20260928 23:06:59.077807 20294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:38833
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:06:59.100668 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:59.178442 20489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:59.178637 20489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:59.178663 20489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:59.179792 20489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:59.179832 20489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:59.179883 20489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:06:59.181349 20489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:59.181582 20489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:59.181798 20489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:59.186435 20495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.186473 20494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.186439 20497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.186487 20489 server_base.cc:1034] running on GCE node
I20260928 23:06:59.186906 20489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:59.187109 20489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:59.188242 20489 hybrid_clock.cc:648] HybridClock initialized: now 1790636819188230 us; error 27 us; skew 500 ppm
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:59.190673 20489 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:06:59.191027 20489 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:59.191085 20489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:59.191239 20489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:59.192209 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "a745420ed099915d0139d62dd3ed798b"
server_key_iv: "7b22190d760854a204d78c3879a74fe1"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.192581 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "a745420ed099915d0139d62dd3ed798b"
server_key_iv: "7b22190d760854a204d78c3879a74fe1"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.193783 20489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:59.194456 20503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.194610 20489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:59.194693 20489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "a745420ed099915d0139d62dd3ed798b"
server_key_iv: "7b22190d760854a204d78c3879a74fe1"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.194743 20489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:59.203353 20489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:59.205083 20489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:59.205189 20489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:59.205336 20489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:59.205550 20489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:59.205577 20489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.205596 20489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:59.205608 20489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.210884 20489 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35443
I20260928 23:06:59.210947 20616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35443 every 8 connection(s)
I20260928 23:06:59.211184 20489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:06:59.214597 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20489
I20260928 23:06:59.214689 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:06:59.214874 16209 external_mini_cluster.cc:1428] Setting key 8d6f6824fab3bb772b13fc07f9c753a1
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:59.220515 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.211840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37653 (local address 127.15.212.66:46921)
0928 23:06:59.212010 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.212047 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.212478 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.212705 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.212708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.212935 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:59.213390 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.213395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.214022 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.214023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.214889 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.214891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.215386 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.215387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.215538 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.216076 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:59.216084 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:59.217637 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:59.219192 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.219198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.219200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.219426 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.219659 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.219659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.219660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.219749 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.220028 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:59.220031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:59.220129 (+ 98us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.220207 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.220311 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260928 23:06:59.220515 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.211859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:46921 (local address 127.15.212.126:37653)
0928 23:06:59.211920 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.211922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.212121 (+ 199us) server_negotiation.cc:407] Connection header received
0928 23:06:59.212578 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.212579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.212589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.212654 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.213462 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.213951 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.215039 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.215305 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.215361 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.217742 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.217743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.217743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:59.217750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.219084 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.219513 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.219515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.219515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.219584 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.219815 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.219817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.219817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.219958 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.220033 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.220255 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.220351 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:06:59.220957 20617 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37653
I20260928 23:06:59.221069 20617 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:59.221235 20617 heartbeater.cc:511] Master 127.15.212.126:37653 requested a full tablet report, sending...
I20260928 23:06:59.221581 20293 ts_manager.cc:194] Registered new tserver with Master: 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c (127.15.212.66:35443)
I20260928 23:06:59.221954 20293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:46921
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:06:59.240754 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:06:59.320174 20622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:59.320393 20622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:59.320431 20622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:59.321563 20622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:59.321626 20622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:59.321668 20622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:06:59.323112 20622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:37653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:59.323366 20622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:59.323585 20622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:06:59.328465 20622 server_base.cc:1034] running on GCE node
W20260928 23:06:59.328413 20628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.328418 20627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.328419 20630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.328810 20622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:59.329064 20622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:59.330209 20622 hybrid_clock.cc:648] HybridClock initialized: now 1790636819330199 us; error 26 us; skew 500 ppm
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:06:59.332608 20622 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:06:59.332963 20622 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:59.333012 20622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:59.333155 20622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:59.334133 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "91ef1c57e1d347ed8ea0927b04d94428"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "9429158bba6e89dbf690d1b9fd6d76bd"
server_key_iv: "1ffc54329c6294c774c8aaa9842f2898"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.334494 20622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "91ef1c57e1d347ed8ea0927b04d94428"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "9429158bba6e89dbf690d1b9fd6d76bd"
server_key_iv: "1ffc54329c6294c774c8aaa9842f2898"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.335740 20622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:06:59.336395 20636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.336556 20622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:59.336642 20622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "91ef1c57e1d347ed8ea0927b04d94428"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "9429158bba6e89dbf690d1b9fd6d76bd"
server_key_iv: "1ffc54329c6294c774c8aaa9842f2898"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.336709 20622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:59.348428 20622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:59.350291 20622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:59.350432 20622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:59.350675 20622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:59.351066 20622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:59.351158 20622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.351207 20622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:59.351230 20622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.356971 20622 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:36639
I20260928 23:06:59.357008 20749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:36639 every 8 connection(s)
I20260928 23:06:59.357260 20622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:06:59 dist-test-slave-x2t8 krb5kdc[20250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636819, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:06:59.364861 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20622
I20260928 23:06:59.364986 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:06:59.365236 16209 external_mini_cluster.cc:1428] Setting key be033fa19044a3f1dcbafb93d7475c97
I20260928 23:06:59.366140 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.357893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:39847 (local address 127.15.212.126:37653)
0928 23:06:59.357949 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.357951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.358124 (+ 173us) server_negotiation.cc:407] Connection header received
0928 23:06:59.358566 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.358567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.358575 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.358629 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.359520 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.360070 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.361039 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.361216 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.361271 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.363110 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.363112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.363112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:59.363118 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.364386 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.365010 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.365011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.365012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.365090 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.365386 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.365387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.365387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.365527 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.365601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.365945 (+ 344us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.366022 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:06:59.366235 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.357867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37653 (local address 127.15.212.67:39847)
0928 23:06:59.358010 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.358049 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.358485 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.358684 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.358687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.358984 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:59.359449 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.359454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.360129 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.360130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.360963 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.360965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.361283 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.361283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.361374 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.361756 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:06:59.361764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:59.363030 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:59.364604 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.364610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.364611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.364914 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.365166 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:06:59.365167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:06:59.365168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:06:59.365305 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:06:59.365605 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:59.365611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:59.365766 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.365899 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.366028 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260928 23:06:59.366696 20750 heartbeater.cc:347] Connected to a master server at 127.15.212.126:37653
I20260928 23:06:59.366780 20750 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:59.366886 20750 heartbeater.cc:511] Master 127.15.212.126:37653 requested a full tablet report, sending...
I20260928 23:06:59.367216 20293 ts_manager.cc:194] Registered new tserver with Master: 91ef1c57e1d347ed8ea0927b04d94428 (127.15.212.67:36639)
I20260928 23:06:59.367539 20293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:39847
I20260928 23:06:59.377171 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:06:59.384230 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.378517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.15.212.126:37653)
0928 23:06:59.378577 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.378580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.378689 (+ 109us) server_negotiation.cc:407] Connection header received
0928 23:06:59.378779 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.378779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.378789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.378841 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.379389 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.379867 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.380622 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.380828 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.380907 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.381994 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.381995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.381995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:06:59.382002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.383140 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.383461 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.383462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.383463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.383516 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:06:59.383715 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:06:59.383716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:06:59.383716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:06:59.383855 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.383952 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.384053 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.384130 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:06:59.385721 20293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:06:59.387423 20293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:06:59.393678 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:36639 (local address 127.0.0.1:46756)
0928 23:06:59.389730 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.389770 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.389834 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.390410 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.390411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.390413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:59.390465 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.390467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.391742 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.391744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.392766 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.392767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.393388 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.393389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.393462 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.393475 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.393547 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.393598 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":54,"threads_started":1}
I20260928 23:06:59.393679 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.15.212.67:36639)
0928 23:06:59.390243 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.390247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.390256 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:06:59.390291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.390293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.390306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.390407 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:59.390529 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.391635 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.392837 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.393309 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.393390 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.393447 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.393560 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.393601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":75,"threads_started":1}
I20260928 23:06:59.393790 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43039 (local address 127.0.0.1:39302)
0928 23:06:59.390071 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.390081 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.390146 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.390503 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.390504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.390506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:59.390540 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.390541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.391367 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.391368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.392598 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.392601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.393550 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.393551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.393626 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.393630 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.393677 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.393718 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260928 23:06:59.393980 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.15.212.65:43039)
0928 23:06:59.390041 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.390047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.390125 (+ 78us) server_negotiation.cc:407] Connection header received
0928 23:06:59.390221 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.390223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.390239 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.390309 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:59.390594 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.391275 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.392682 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.393398 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.393469 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.393517 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.393860 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.393905 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260928 23:06:59.394802 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35443 (local address 127.0.0.1:41884)
0928 23:06:59.390059 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.390070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.390130 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.390325 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.390329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.390331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:59.390401 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.390403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.392480 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.392481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.393543 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.393545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.394549 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.394550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.394628 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.394641 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.394700 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.394742 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":49,"threads_started":1}
I20260928 23:06:59.394723 20684 tablet_service.cc:1467] Processing CreateTablet for tablet 896526ec578f43828965fbaf36a5de66 (DEFAULT_TABLE table=test-table [id=c611b8c193ec45e9a0f299ac37aa014b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:59.394872 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.389890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.15.212.66:35443)
0928 23:06:59.390085 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.390090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.390117 (+ 27us) server_negotiation.cc:407] Connection header received
0928 23:06:59.390193 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.390196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.390211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.390474 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:59.390581 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.391649 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.393630 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.394367 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.394467 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.394520 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.394738 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.394789 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260928 23:06:59.394814 20418 tablet_service.cc:1467] Processing CreateTablet for tablet 896526ec578f43828965fbaf36a5de66 (DEFAULT_TABLE table=test-table [id=c611b8c193ec45e9a0f299ac37aa014b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:59.395090 20684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896526ec578f43828965fbaf36a5de66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:59.395113 20418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896526ec578f43828965fbaf36a5de66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:59.395804 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 896526ec578f43828965fbaf36a5de66 (DEFAULT_TABLE table=test-table [id=c611b8c193ec45e9a0f299ac37aa014b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:06:59.396080 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896526ec578f43828965fbaf36a5de66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:59.397699 20764 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Bootstrap starting.
I20260928 23:06:59.398267 20764 tablet_bootstrap.cc:654] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:59.398334 20765 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Bootstrap starting.
I20260928 23:06:59.398995 20764 log.cc:824] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:59.399066 20765 tablet_bootstrap.cc:654] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:59.399361 20766 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Bootstrap starting.
I20260928 23:06:59.399642 20765 log.cc:824] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:59.399955 20766 tablet_bootstrap.cc:654] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:59.400377 20765 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: No bootstrap required, opened a new log
I20260928 23:06:59.400497 20765 ts_tablet_manager.cc:1397] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:59.400557 20766 log.cc:824] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:59.401250 20766 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: No bootstrap required, opened a new log
I20260928 23:06:59.401258 20764 tablet_bootstrap.cc:492] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: No bootstrap required, opened a new log
I20260928 23:06:59.401316 20764 ts_tablet_manager.cc:1397] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 23:06:59.401376 20766 ts_tablet_manager.cc:1397] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:06:59.402175 20765 raft_consensus.cc:348] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.402318 20765 raft_consensus.cc:374] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:59.402377 20765 raft_consensus.cc:729] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb3fa3d606b4532b54f52744ffcaff5, State: Initialized, Role: FOLLOWER
I20260928 23:06:59.402503 20765 consensus_queue.cc:260] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.402781 20765 ts_tablet_manager.cc:1428] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:59.402911 20484 heartbeater.cc:503] Master 127.15.212.126:37653 was elected leader, sending a full tablet report...
I20260928 23:06:59.403265 20766 raft_consensus.cc:348] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.403265 20764 raft_consensus.cc:348] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.403421 20766 raft_consensus.cc:374] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:59.403420 20764 raft_consensus.cc:374] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:59.403455 20764 raft_consensus.cc:729] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ef1c57e1d347ed8ea0927b04d94428, State: Initialized, Role: FOLLOWER
I20260928 23:06:59.403455 20766 raft_consensus.cc:729] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c, State: Initialized, Role: FOLLOWER
I20260928 23:06:59.403580 20766 consensus_queue.cc:260] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.403612 20764 consensus_queue.cc:260] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.403853 20764 ts_tablet_manager.cc:1428] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:06:59.403853 20766 ts_tablet_manager.cc:1428] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:06:59.403972 20617 heartbeater.cc:503] Master 127.15.212.126:37653 was elected leader, sending a full tablet report...
I20260928 23:06:59.403985 20750 heartbeater.cc:503] Master 127.15.212.126:37653 was elected leader, sending a full tablet report...
W20260928 23:06:59.461714 20618 tablet.cc:2369] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:59.566159 20771 raft_consensus.cc:482] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:06:59.566313 20771 raft_consensus.cc:504] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.566673 20771 leader_election.cc:288] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ef1c57e1d347ed8ea0927b04d94428 (127.15.212.67:36639), 0bb3fa3d606b4532b54f52744ffcaff5 (127.15.212.65:43039)
W20260928 23:06:59.567973 20485 tablet.cc:2369] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:59.569628 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.566984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:52509 (local address 127.15.212.67:36639)
0928 23:06:59.567042 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.567044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.567058 (+ 14us) server_negotiation.cc:407] Connection header received
0928 23:06:59.567103 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.567104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.567113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.567213 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:59.567350 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.567813 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.568916 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.569293 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.569380 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.569389 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.569523 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.569569 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:06:59.569629 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.566865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:36639 (local address 127.15.212.66:52509)
0928 23:06:59.566932 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.566946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.567017 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.567221 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.567222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.567224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:59.567276 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.567278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.567912 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.567913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.568824 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.568826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.569377 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.569378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.569446 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.569450 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.569519 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.569556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 23:06:59.569730 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.567042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34195 (local address 127.15.212.65:43039)
0928 23:06:59.567087 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.567089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.567215 (+ 126us) server_negotiation.cc:407] Connection header received
0928 23:06:59.567273 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.567274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.567283 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.567345 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:06:59.567505 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.568058 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.569032 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.569402 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.569485 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.569492 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.569627 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.569671 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:06:59.569741 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.566975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43039 (local address 127.15.212.66:34195)
0928 23:06:59.567147 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.567155 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.567216 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.567360 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.567361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.567363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:06:59.567432 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.567434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.568167 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.568168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.568945 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:06:59.568947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.569484 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.569485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:06:59.569556 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.569559 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.569621 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.569661 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260928 23:06:59.570096 20704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896526ec578f43828965fbaf36a5de66" candidate_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ef1c57e1d347ed8ea0927b04d94428" is_pre_election: true
I20260928 23:06:59.570102 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896526ec578f43828965fbaf36a5de66" candidate_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" is_pre_election: true
I20260928 23:06:59.570236 20704 raft_consensus.cc:2454] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c in term 0.
I20260928 23:06:59.570252 20438 raft_consensus.cc:2454] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c in term 0.
I20260928 23:06:59.570436 20505 leader_election.cc:302] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bb3fa3d606b4532b54f52744ffcaff5, 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c; no voters:
I20260928 23:06:59.570551 20771 raft_consensus.cc:2781] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:06:59.570606 20771 raft_consensus.cc:482] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:06:59.570622 20771 raft_consensus.cc:3037] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:59.571296 20771 raft_consensus.cc:504] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.571431 20771 leader_election.cc:288] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [CANDIDATE]: Term 1 election: Requested vote from peers 91ef1c57e1d347ed8ea0927b04d94428 (127.15.212.67:36639), 0bb3fa3d606b4532b54f52744ffcaff5 (127.15.212.65:43039)
I20260928 23:06:59.571614 20704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896526ec578f43828965fbaf36a5de66" candidate_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ef1c57e1d347ed8ea0927b04d94428"
I20260928 23:06:59.571628 20438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896526ec578f43828965fbaf36a5de66" candidate_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bb3fa3d606b4532b54f52744ffcaff5"
I20260928 23:06:59.571702 20704 raft_consensus.cc:3037] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:59.571743 20438 raft_consensus.cc:3037] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:59.572438 20438 raft_consensus.cc:2454] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c in term 1.
I20260928 23:06:59.572501 20704 raft_consensus.cc:2454] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c in term 1.
I20260928 23:06:59.572672 20505 leader_election.cc:302] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c, 91ef1c57e1d347ed8ea0927b04d94428; no voters:
I20260928 23:06:59.572788 20771 raft_consensus.cc:2781] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:59.572996 20771 raft_consensus.cc:686] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 1 LEADER]: Becoming Leader. State: Replica: 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c, State: Running, Role: LEADER
I20260928 23:06:59.573099 20771 consensus_queue.cc:237] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } }
I20260928 23:06:59.573866 20292 catalog_manager.cc:5495] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c (127.15.212.66). New cstate: current_term: 1 leader_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4cb8742c3f42c9b4cf6c5b5f9a0a4c" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35443 } health_report { overall_health: HEALTHY } } }
W20260928 23:06:59.607803 20751 tablet.cc:2369] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:06:59.621673 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.619366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.15.212.66:35443)
0928 23:06:59.619446 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.619449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.619477 (+ 28us) server_negotiation.cc:407] Connection header received
0928 23:06:59.619515 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.619515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.619533 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.619607 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:06:59.619753 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.620265 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.620984 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:06:59.621187 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:06:59.621248 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:06:59.621359 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:06:59.621461 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:06:59.621558 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.621575 (+ 17us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.621592 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 23:06:59.623384 20438 raft_consensus.cc:1261] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 1 FOLLOWER]: Refusing update from remote peer 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:59.623450 20704 raft_consensus.cc:1261] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 1 FOLLOWER]: Refusing update from remote peer 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:06:59.623737 20774 consensus_queue.cc:1035] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bb3fa3d606b4532b54f52744ffcaff5" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:59.623870 20771 consensus_queue.cc:1035] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ef1c57e1d347ed8ea0927b04d94428" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 36639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:06:59.626235 20777 mvcc.cc:205] Tried to move back new op lower bound from 7334448413174853632 to 7334448412971978752. Current Snapshot: MvccSnapshot[applied={T|T < 7334448413174853632}]
I20260928 23:06:59.626783 20778 mvcc.cc:205] Tried to move back new op lower bound from 7334448413174853632 to 7334448412971978752. Current Snapshot: MvccSnapshot[applied={T|T < 7334448413174853632}]
I20260928 23:06:59.628816 20294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:59.628917 20294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:06:59.629671 20294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74146704c52747819abd66122c0bf5d8: Sending DeleteTablet for 3 replicas of tablet 896526ec578f43828965fbaf36a5de66
I20260928 23:06:59.629997 20684 tablet_service.cc:1514] Processing DeleteTablet for tablet 896526ec578f43828965fbaf36a5de66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:59 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:46756
I20260928 23:06:59.630033 20418 tablet_service.cc:1514] Processing DeleteTablet for tablet 896526ec578f43828965fbaf36a5de66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:59 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:39302
I20260928 23:06:59.630249 20787 tablet_replica.cc:330] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: stopping tablet replica
I20260928 23:06:59.630231 20551 tablet_service.cc:1514] Processing DeleteTablet for tablet 896526ec578f43828965fbaf36a5de66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:06:59 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:41884
I20260928 23:06:59.630352 20787 raft_consensus.cc:2229] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:59.630455 20787 raft_consensus.cc:2258] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:59.630471 20788 tablet_replica.cc:330] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: stopping tablet replica
I20260928 23:06:59.630558 20788 raft_consensus.cc:2229] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:06:59.630671 20786 tablet_replica.cc:330] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: stopping tablet replica
I20260928 23:06:59.630756 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20356
I20260928 23:06:59.630774 20786 raft_consensus.cc:2229] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:06:59.630862 20786 raft_consensus.cc:2258] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:59.631035 20787 ts_tablet_manager.cc:1905] T 896526ec578f43828965fbaf36a5de66 P 0bb3fa3d606b4532b54f52744ffcaff5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:59.631299 20786 ts_tablet_manager.cc:1905] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:59.633203 20786 ts_tablet_manager.cc:1918] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:06:59.633262 20786 log.cc:1196] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/896526ec578f43828965fbaf36a5de66
I20260928 23:06:59.633517 20786 ts_tablet_manager.cc:1939] T 896526ec578f43828965fbaf36a5de66 P 91ef1c57e1d347ed8ea0927b04d94428: Deleting consensus metadata
I20260928 23:06:59.633651 20788 raft_consensus.cc:2258] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:06:59.634042 20788 ts_tablet_manager.cc:1905] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:06:59.635816 20788 ts_tablet_manager.cc:1918] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:06:59.635892 20788 log.cc:1196] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/896526ec578f43828965fbaf36a5de66
I20260928 23:06:59.636248 20788 ts_tablet_manager.cc:1939] T 896526ec578f43828965fbaf36a5de66 P 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c: Deleting consensus metadata
I20260928 23:06:59.636716 20278 catalog_manager.cc:4841] TS 0e4cb8742c3f42c9b4cf6c5b5f9a0a4c (127.15.212.66:35443): tablet 896526ec578f43828965fbaf36a5de66 (table test-table [id=c611b8c193ec45e9a0f299ac37aa014b]) successfully deleted
W20260928 23:06:59.637483 20279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:43039 (error 108)
I20260928 23:06:59.637670 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20489
I20260928 23:06:59.638432 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.637818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43039 (local address 127.0.0.1:39316)
0928 23:06:59.638371 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.638406 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:43039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536}
W20260928 23:06:59.638586 20279 catalog_manager.cc:4584] TS 0bb3fa3d606b4532b54f52744ffcaff5 (127.15.212.65:43039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 896526ec578f43828965fbaf36a5de66: Network error: Client connection negotiation failed: client connection to 127.15.212.65:43039: connect: Connection refused (error 111)
I20260928 23:06:59.639554 20279 catalog_manager.cc:4841] TS 91ef1c57e1d347ed8ea0927b04d94428 (127.15.212.67:36639): tablet 896526ec578f43828965fbaf36a5de66 (table test-table [id=c611b8c193ec45e9a0f299ac37aa014b]) successfully deleted
I20260928 23:06:59.643368 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20622
I20260928 23:06:59.647692 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20263
2026-09-28T23:06:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2025 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T23:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:06:59Z Disabled control of system clock
I20260928 23:06:59.674626 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:35299
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:35299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 23:06:59.759717 20793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:59.759895 20793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:59.759915 20793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:59.760979 20793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:06:59.761014 20793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:59.761026 20793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:59.761039 20793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:06:59.761050 20793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:06:59.762257 20793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:35299
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:35299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:59.762454 20793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:59.762670 20793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:59.767202 20799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.767222 20801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.767584 20793 server_base.cc:1034] running on GCE node
W20260928 23:06:59.767222 20798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.767994 20793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:59.768283 20793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:59.769459 20793 hybrid_clock.cc:648] HybridClock initialized: now 1790636819769428 us; error 31 us; skew 500 ppm
I20260928 23:06:59.770551 20793 webserver.cc:460] Webserver started at http://127.15.212.126:36749/ using document root <none> and password file <none>
I20260928 23:06:59.770828 20793 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:59.770890 20793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:59.771028 20793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:59.771983 20793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "260d6e1614a04e17b90c0b377cd811ab"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "bdb23b0fcdb83c34f4b331c5bcf9571d"
server_key_iv: "5dea2ebb91c757d83a24479bd40262b2"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.772322 20793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "260d6e1614a04e17b90c0b377cd811ab"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "bdb23b0fcdb83c34f4b331c5bcf9571d"
server_key_iv: "5dea2ebb91c757d83a24479bd40262b2"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.773654 20793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:06:59.774322 20807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.774477 20793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:06:59.774569 20793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "260d6e1614a04e17b90c0b377cd811ab"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "bdb23b0fcdb83c34f4b331c5bcf9571d"
server_key_iv: "5dea2ebb91c757d83a24479bd40262b2"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.774647 20793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:59.777802 20793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:59.778033 20793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:59.778164 20793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:59.781744 20793 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:35299
I20260928 23:06:59.781775 20859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:35299 every 8 connection(s)
I20260928 23:06:59.782066 20793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:06:59.782634 20860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:06:59.785192 20860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: Bootstrap starting.
I20260928 23:06:59.785681 20860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: Neither blocks nor log segments found. Creating new log.
I20260928 23:06:59.785913 20860 log.cc:824] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: Log is configured to *not* fsync() on all Append() calls
I20260928 23:06:59.786502 20860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: No bootstrap required, opened a new log
I20260928 23:06:59.787951 20860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } }
I20260928 23:06:59.788084 20860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:06:59.788106 20860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260d6e1614a04e17b90c0b377cd811ab, State: Initialized, Role: FOLLOWER
I20260928 23:06:59.788236 20860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } }
I20260928 23:06:59.788312 20860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:06:59.788334 20860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:06:59.788367 20860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:06:59.788642 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20793
I20260928 23:06:59.788748 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:06:59.788940 16209 external_mini_cluster.cc:1428] Setting key 97981125e792161ede991bef96d37d37
I20260928 23:06:59.789090 20860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } }
I20260928 23:06:59.789196 20860 leader_election.cc:302] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 260d6e1614a04e17b90c0b377cd811ab; no voters:
I20260928 23:06:59.789356 20860 leader_election.cc:288] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:06:59.789408 20863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:06:59.789542 20863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [term 1 LEADER]: Becoming Leader. State: Replica: 260d6e1614a04e17b90c0b377cd811ab, State: Running, Role: LEADER
I20260928 23:06:59.789609 20860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:06:59.789610 20863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } }
I20260928 23:06:59.789978 20863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 260d6e1614a04e17b90c0b377cd811ab. Latest consensus state: current_term: 1 leader_uuid: "260d6e1614a04e17b90c0b377cd811ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } } }
I20260928 23:06:59.790053 20863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:59.790213 20865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "260d6e1614a04e17b90c0b377cd811ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "260d6e1614a04e17b90c0b377cd811ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 35299 } } }
I20260928 23:06:59.790261 20870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:06:59.790314 20865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:06:59.790835 20870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:06:59.792394 20870 catalog_manager.cc:1347] Generated new cluster ID: 945cc7050bf84b1bbfa629feaa9d3b78
I20260928 23:06:59.792520 20870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:06:59.793107 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.789438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.15.212.126:35299)
0928 23:06:59.789771 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.789779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.791614 (+ 1835us) server_negotiation.cc:407] Connection header received
0928 23:06:59.791736 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.791742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.791786 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.791931 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.792654 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.792657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.792657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:06:59.792662 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.792716 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:06:59.792727 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.792799 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.792873 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.792932 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260928 23:06:59.822206 20870 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:06:59.822360 20870 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:06:59.828058 20870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: Generated new TSK 0
I20260928 23:06:59.828231 20870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:06:59.836243 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--builtin_ntp_servers=127.15.212.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 23:06:59.914264 20884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:06:59.914462 20884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:06:59.914484 20884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:06:59.915652 20884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:06:59.915690 20884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:06:59.915715 20884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:06:59.917002 20884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:06:59.917268 20884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:06:59.917562 20884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:06:59.922537 20892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:06:59.922483 20889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.922521 20884 server_base.cc:1034] running on GCE node
W20260928 23:06:59.922485 20890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:06:59.922895 20884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:06:59.923111 20884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:06:59.924257 20884 hybrid_clock.cc:648] HybridClock initialized: now 1790636819924239 us; error 35 us; skew 500 ppm
I20260928 23:06:59.930980 20884 webserver.cc:460] Webserver started at http://127.15.212.65:36945/ using document root <none> and password file <none>
I20260928 23:06:59.931210 20884 fs_manager.cc:359] Metadata directory not provided
I20260928 23:06:59.931255 20884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:06:59.931373 20884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:06:59.932474 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "1ba480bcb6824dcd8babe79cd58a4614"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "6052dff202b6839cbfdd8eaa47bb7e9e"
server_key_iv: "b31898dffa98e3af4af91d9d49333520"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.932859 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "1ba480bcb6824dcd8babe79cd58a4614"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "6052dff202b6839cbfdd8eaa47bb7e9e"
server_key_iv: "b31898dffa98e3af4af91d9d49333520"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.934109 20884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:06:59.934772 20898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.934927 20884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:06:59.935024 20884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "1ba480bcb6824dcd8babe79cd58a4614"
format_stamp: "Formatted at 2026-09-28 23:06:59 on dist-test-slave-x2t8"
server_key: "6052dff202b6839cbfdd8eaa47bb7e9e"
server_key_iv: "b31898dffa98e3af4af91d9d49333520"
server_key_version: "encryptionkey@0"
I20260928 23:06:59.935106 20884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:06:59.938409 20884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:06:59.938651 20884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:06:59.938796 20884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:06:59.938987 20884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:06:59.939266 20884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:06:59.939314 20884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.939350 20884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:06:59.939379 20884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:06:59.945078 20884 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:41523
I20260928 23:06:59.945111 21011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:41523 every 8 connection(s)
I20260928 23:06:59.945415 20884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:06:59.948266 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.946090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:34203 (local address 127.15.212.126:35299)
0928 23:06:59.946161 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:06:59.946163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:06:59.946362 (+ 199us) server_negotiation.cc:407] Connection header received
0928 23:06:59.946838 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:06:59.946839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:06:59.946847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:06:59.946897 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:06:59.948014 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:06:59.948015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:06:59.948016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:06:59.948021 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:06:59.948036 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:06:59.948041 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:06:59.948091 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:06:59.948181 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 23:06:59.948197 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:06:59.948473 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:06:59.946078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:35299 (local address 127.15.212.65:34203)
0928 23:06:59.946260 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 23:06:59.946289 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 23:06:59.946732 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:06:59.946960 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:06:59.946976 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:06:59.946990 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:06:59.947849 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:06:59.947849 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:06:59.947874 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:06:59.947900 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:06:59.948093 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:06:59.948099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:06:59.948102 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:06:59.948192 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 23:06:59.948248 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260928 23:06:59.948956 21012 heartbeater.cc:347] Connected to a master server at 127.15.212.126:35299
I20260928 23:06:59.949023 21012 heartbeater.cc:464] Registering TS with master...
I20260928 23:06:59.949120 21012 heartbeater.cc:511] Master 127.15.212.126:35299 requested a full tablet report, sending...
I20260928 23:06:59.949546 20824 ts_manager.cc:194] Registered new tserver with Master: 1ba480bcb6824dcd8babe79cd58a4614 (127.15.212.65:41523)
I20260928 23:06:59.950608 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 20884
I20260928 23:06:59.950691 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:06:59.950826 16209 external_mini_cluster.cc:1428] Setting key 4a78f5d8289ca9b695f7a4806d9154b4
I20260928 23:06:59.951552 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--builtin_ntp_servers=127.15.212.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 23:07:00.044636 21015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:00.044833 21015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:00.044865 21015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:00.046001 21015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:00.046061 21015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:00.046103 21015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:00.047405 21015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:00.047640 21015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:00.047920 21015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:00.052683 21021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.052685 21023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.052732 21015 server_base.cc:1034] running on GCE node
W20260928 23:07:00.052732 21020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.053085 21015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:00.053330 21015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:00.054461 21015 hybrid_clock.cc:648] HybridClock initialized: now 1790636820054447 us; error 31 us; skew 500 ppm
I20260928 23:07:00.055672 21015 webserver.cc:460] Webserver started at http://127.15.212.66:33809/ using document root <none> and password file <none>
I20260928 23:07:00.055900 21015 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:00.055943 21015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:00.056118 21015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:00.057166 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "7dc9dcb89d154556aef028353359f7ec"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "a8f4038e76ec0af37396e6209534fb13"
server_key_iv: "291946b2521eb3dfb1430ebfb9a9f84c"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.057519 21015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "7dc9dcb89d154556aef028353359f7ec"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "a8f4038e76ec0af37396e6209534fb13"
server_key_iv: "291946b2521eb3dfb1430ebfb9a9f84c"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.058755 21015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:07:00.059437 21029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.059636 21015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:00.059728 21015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "7dc9dcb89d154556aef028353359f7ec"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "a8f4038e76ec0af37396e6209534fb13"
server_key_iv: "291946b2521eb3dfb1430ebfb9a9f84c"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.059798 21015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:00.063019 21015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:00.063249 21015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:00.063377 21015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:00.063573 21015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:00.063903 21015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:00.063951 21015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.063993 21015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:00.064023 21015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.069667 21015 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:43395
I20260928 23:07:00.069734 21142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:43395 every 8 connection(s)
I20260928 23:07:00.070019 21015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:00.072726 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.070644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:57913 (local address 127.15.212.126:35299)
0928 23:07:00.070725 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.070728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.070921 (+ 193us) server_negotiation.cc:407] Connection header received
0928 23:07:00.071333 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.071334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.071339 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.071390 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.072537 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.072539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.072539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.072543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.072556 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.072560 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.072605 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.072662 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.072674 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:07:00.072901 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.070617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:35299 (local address 127.15.212.66:57913)
0928 23:07:00.070785 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.070827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.071253 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.071436 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.071439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.071446 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.072344 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.072345 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.072369 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.072394 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.072605 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.072607 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.072610 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.072661 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.072705 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260928 23:07:00.073372 21143 heartbeater.cc:347] Connected to a master server at 127.15.212.126:35299
I20260928 23:07:00.073453 21143 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:00.073556 21143 heartbeater.cc:511] Master 127.15.212.126:35299 requested a full tablet report, sending...
I20260928 23:07:00.073855 20824 ts_manager.cc:194] Registered new tserver with Master: 7dc9dcb89d154556aef028353359f7ec (127.15.212.66:43395)
I20260928 23:07:00.075661 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21015
I20260928 23:07:00.075759 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:00.075932 16209 external_mini_cluster.cc:1428] Setting key 82de29a45cc620d959bccc0abf1ed139
I20260928 23:07:00.076768 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--builtin_ntp_servers=127.15.212.84:46681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 23:07:00.158749 21146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:00.158950 21146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:00.158983 21146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:00.160106 21146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:00.160171 21146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:00.160211 21146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:00.161587 21146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:35299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:00.161837 21146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:00.162159 21146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:00.167017 21152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.167043 21154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.167052 21151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.167255 21146 server_base.cc:1034] running on GCE node
I20260928 23:07:00.167455 21146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:00.167682 21146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:00.168833 21146 hybrid_clock.cc:648] HybridClock initialized: now 1790636820168839 us; error 61 us; skew 500 ppm
I20260928 23:07:00.170077 21146 webserver.cc:460] Webserver started at http://127.15.212.67:34171/ using document root <none> and password file <none>
I20260928 23:07:00.170316 21146 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:00.170380 21146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:00.170528 21146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:00.171552 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "1b39330aa0284d959ffe177d9e1fd282"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "fccfb2c4d88013185cec530e2eb5d06d"
server_key_iv: "a93ee5eeba761f4bd92e632adfa696f2"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.171895 21146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "1b39330aa0284d959ffe177d9e1fd282"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "fccfb2c4d88013185cec530e2eb5d06d"
server_key_iv: "a93ee5eeba761f4bd92e632adfa696f2"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.173184 21146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:00.173832 21160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.173986 21146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:00.174083 21146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "1b39330aa0284d959ffe177d9e1fd282"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "fccfb2c4d88013185cec530e2eb5d06d"
server_key_iv: "a93ee5eeba761f4bd92e632adfa696f2"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.174160 21146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:00.177428 21146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:00.177662 21146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:00.177810 21146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:00.178011 21146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:00.178292 21146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:00.178341 21146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.178380 21146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:00.178409 21146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.183823 21146 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:32957
I20260928 23:07:00.183880 21273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:32957 every 8 connection(s)
I20260928 23:07:00.184150 21146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:00.186803 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.184803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:42457 (local address 127.15.212.126:35299)
0928 23:07:00.184878 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.184880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.185072 (+ 192us) server_negotiation.cc:407] Connection header received
0928 23:07:00.185467 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.185468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.185472 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.185522 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.186601 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.186602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.186602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.186606 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.186624 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.186629 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.186673 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.186742 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.186754 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 23:07:00.186983 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.184768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:35299 (local address 127.15.212.67:42457)
0928 23:07:00.184956 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.184986 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.185389 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.185578 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.185581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.185589 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.186430 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.186431 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.186457 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.186515 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.186674 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.186676 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.186686 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.186738 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.186782 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260928 23:07:00.187391 21274 heartbeater.cc:347] Connected to a master server at 127.15.212.126:35299
I20260928 23:07:00.187482 21274 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:00.187620 21274 heartbeater.cc:511] Master 127.15.212.126:35299 requested a full tablet report, sending...
I20260928 23:07:00.187954 20824 ts_manager.cc:194] Registered new tserver with Master: 1b39330aa0284d959ffe177d9e1fd282 (127.15.212.67:32957)
I20260928 23:07:00.191154 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21146
I20260928 23:07:00.191233 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:00.191370 16209 external_mini_cluster.cc:1428] Setting key d6e598eef2aa393276c67924049ffa47
I20260928 23:07:00.192149 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:07:00.194353 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.193321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.15.212.126:35299)
0928 23:07:00.193373 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.193374 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.193509 (+ 135us) server_negotiation.cc:407] Connection header received
0928 23:07:00.193548 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.193549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.193553 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.193606 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.194164 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.194165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.194165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.194168 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.194177 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.194180 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.194223 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.194253 (+ 30us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.194264 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:00.195654 20824 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:00.197263 20824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:00.201027 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:41523 (local address 127.0.0.1:55774)
0928 23:07:00.199372 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.199410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.199489 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.199737 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.199741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.199747 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.200661 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.200661 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.200672 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.200687 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.200885 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.200890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.200893 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.200954 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.200967 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":82,"threads_started":1}
I20260928 23:07:00.201052 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.15.212.65:41523)
0928 23:07:00.199599 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.199608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.199622 (+ 14us) server_negotiation.cc:407] Connection header received
0928 23:07:00.199643 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.199645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.199651 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.199723 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.200765 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.200773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.200773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.200777 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.200807 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.200830 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.200891 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.200963 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.200978 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":111,"threads_started":1}
I20260928 23:07:00.201416 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:43395 (local address 127.0.0.1:60326)
0928 23:07:00.199643 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.199655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.199708 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.199878 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.199879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.199882 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.200812 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.200813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.200820 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.200832 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.201275 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.201276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.201277 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.201343 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.201356 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":41,"threads_started":1}
I20260928 23:07:00.201697 20946 tablet_service.cc:1467] Processing CreateTablet for tablet 6e177e488997452982af9ce6d9aea196 (DEFAULT_TABLE table=test-table [id=1f4b8af08e7942ec95da94efffd82542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:00.201953 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e177e488997452982af9ce6d9aea196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.204325 21288 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: Bootstrap starting.
I20260928 23:07:00.204479 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.15.212.66:43395)
0928 23:07:00.199625 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.199629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.199761 (+ 132us) server_negotiation.cc:407] Connection header received
0928 23:07:00.199789 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.199790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.199798 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.199877 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.200915 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.200923 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.200924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.200928 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.200959 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.200989 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.201078 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.204324 (+ 3246us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.204342 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260928 23:07:00.205001 21288 tablet_bootstrap.cc:654] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.204996 21077 tablet_service.cc:1467] Processing CreateTablet for tablet 6e177e488997452982af9ce6d9aea196 (DEFAULT_TABLE table=test-table [id=1f4b8af08e7942ec95da94efffd82542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:00.205295 21077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e177e488997452982af9ce6d9aea196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.205300 21288 log.cc:824] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.205931 21288 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: No bootstrap required, opened a new log
I20260928 23:07:00.206002 21288 ts_tablet_manager.cc:1397] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:00.207698 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:32957 (local address 127.0.0.1:48550)
0928 23:07:00.200198 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.200206 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.200266 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.200406 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.200407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.200409 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.201338 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.201338 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.201343 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.201352 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.204661 (+ 3309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.204662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.204663 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.207623 (+ 2960us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.207637 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":84,"threads_started":1}
I20260928 23:07:00.207692 21290 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Bootstrap starting.
I20260928 23:07:00.207933 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.199440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.15.212.67:32957)
0928 23:07:00.199692 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.199695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.200265 (+ 570us) server_negotiation.cc:407] Connection header received
0928 23:07:00.200324 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.200328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.200336 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.200405 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.204499 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.204505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.204506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.204510 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.204551 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.204573 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.204636 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.207843 (+ 3207us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.207859 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":77,"threads_started":1}
I20260928 23:07:00.207929 21288 raft_consensus.cc:348] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.208092 21288 raft_consensus.cc:374] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.208142 21288 raft_consensus.cc:729] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba480bcb6824dcd8babe79cd58a4614, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.208284 21290 tablet_bootstrap.cc:654] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.208271 21288 consensus_queue.cc:260] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.208575 21012 heartbeater.cc:503] Master 127.15.212.126:35299 was elected leader, sending a full tablet report...
I20260928 23:07:00.208650 21290 log.cc:824] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.208559 21208 tablet_service.cc:1467] Processing CreateTablet for tablet 6e177e488997452982af9ce6d9aea196 (DEFAULT_TABLE table=test-table [id=1f4b8af08e7942ec95da94efffd82542]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:00.208578 21288 ts_tablet_manager.cc:1428] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260928 23:07:00.208880 21208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e177e488997452982af9ce6d9aea196. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.209564 21290 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: No bootstrap required, opened a new log
I20260928 23:07:00.209643 21290 ts_tablet_manager.cc:1397] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:00.211372 21290 raft_consensus.cc:348] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.211506 21290 raft_consensus.cc:374] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.211547 21290 raft_consensus.cc:729] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dc9dcb89d154556aef028353359f7ec, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.211545 21293 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Bootstrap starting.
I20260928 23:07:00.211673 21290 consensus_queue.cc:260] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.211943 21143 heartbeater.cc:503] Master 127.15.212.126:35299 was elected leader, sending a full tablet report...
I20260928 23:07:00.211941 21290 ts_tablet_manager.cc:1428] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:07:00.212069 21293 tablet_bootstrap.cc:654] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.212347 21293 log.cc:824] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.213066 21293 tablet_bootstrap.cc:492] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: No bootstrap required, opened a new log
I20260928 23:07:00.213135 21293 ts_tablet_manager.cc:1397] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:00.214809 21293 raft_consensus.cc:348] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.214937 21293 raft_consensus.cc:374] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.214979 21293 raft_consensus.cc:729] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b39330aa0284d959ffe177d9e1fd282, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.215102 21293 consensus_queue.cc:260] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.215328 21293 ts_tablet_manager.cc:1428] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:07:00.215364 21274 heartbeater.cc:503] Master 127.15.212.126:35299 was elected leader, sending a full tablet report...
W20260928 23:07:00.320550 21144 tablet.cc:2369] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:00.349949 21291 raft_consensus.cc:482] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:00.350071 21291 raft_consensus.cc:504] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.350361 21291 leader_election.cc:288] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dc9dcb89d154556aef028353359f7ec (127.15.212.66:43395), 1b39330aa0284d959ffe177d9e1fd282 (127.15.212.67:32957)
I20260928 23:07:00.357678 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.350644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:51699 (local address 127.15.212.67:32957)
0928 23:07:00.350693 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.350695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.350837 (+ 142us) server_negotiation.cc:407] Connection header received
0928 23:07:00.350891 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.350892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.350898 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.350965 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.357435 (+ 6470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.357437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.357437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.357442 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.357464 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.357476 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.357546 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.357582 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.357599 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 23:07:00.357712 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.350553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:32957 (local address 127.15.212.65:51699)
0928 23:07:00.350764 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.350772 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.350834 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.350968 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.350969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.350971 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.357326 (+ 6355us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.357327 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.357334 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.357347 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.357535 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.357535 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.357536 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.357584 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.357599 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":84,"thread_start_us":85,"threads_started":1}
I20260928 23:07:00.357720 21287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.350653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44131 (local address 127.15.212.66:43395)
0928 23:07:00.350703 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.350705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.350713 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:00.350738 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.350743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.350749 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.350823 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.357493 (+ 6670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.357494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.357495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.357499 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.357514 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.357519 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.357579 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.357626 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.357650 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 23:07:00.357720 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.350552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:43395 (local address 127.15.212.65:44131)
0928 23:07:00.350611 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.350623 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.350659 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.350832 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.350834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.350836 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:00.357365 (+ 6529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:00.357366 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:00.357373 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:00.357384 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:00.357575 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:00.357576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:00.357577 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:00.357627 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 23:07:00.357642 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:07:00.358107 21228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e177e488997452982af9ce6d9aea196" candidate_uuid: "1ba480bcb6824dcd8babe79cd58a4614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b39330aa0284d959ffe177d9e1fd282" is_pre_election: true
I20260928 23:07:00.358278 21228 raft_consensus.cc:2454] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ba480bcb6824dcd8babe79cd58a4614 in term 0.
I20260928 23:07:00.358248 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e177e488997452982af9ce6d9aea196" candidate_uuid: "1ba480bcb6824dcd8babe79cd58a4614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc9dcb89d154556aef028353359f7ec" is_pre_election: true
I20260928 23:07:00.358366 21097 raft_consensus.cc:2454] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ba480bcb6824dcd8babe79cd58a4614 in term 0.
I20260928 23:07:00.358517 20902 leader_election.cc:302] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b39330aa0284d959ffe177d9e1fd282, 1ba480bcb6824dcd8babe79cd58a4614; no voters:
I20260928 23:07:00.358660 21291 raft_consensus.cc:2781] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:00.358717 21291 raft_consensus.cc:482] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:00.358739 21291 raft_consensus.cc:3037] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.359412 21291 raft_consensus.cc:504] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.359562 21291 leader_election.cc:288] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [CANDIDATE]: Term 1 election: Requested vote from peers 7dc9dcb89d154556aef028353359f7ec (127.15.212.66:43395), 1b39330aa0284d959ffe177d9e1fd282 (127.15.212.67:32957)
I20260928 23:07:00.359683 21097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e177e488997452982af9ce6d9aea196" candidate_uuid: "1ba480bcb6824dcd8babe79cd58a4614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dc9dcb89d154556aef028353359f7ec"
I20260928 23:07:00.359712 21228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e177e488997452982af9ce6d9aea196" candidate_uuid: "1ba480bcb6824dcd8babe79cd58a4614" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b39330aa0284d959ffe177d9e1fd282"
I20260928 23:07:00.359776 21228 raft_consensus.cc:3037] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.359768 21097 raft_consensus.cc:3037] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.360698 21097 raft_consensus.cc:2454] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ba480bcb6824dcd8babe79cd58a4614 in term 1.
I20260928 23:07:00.360698 21228 raft_consensus.cc:2454] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ba480bcb6824dcd8babe79cd58a4614 in term 1.
I20260928 23:07:00.360847 20902 leader_election.cc:302] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b39330aa0284d959ffe177d9e1fd282, 1ba480bcb6824dcd8babe79cd58a4614; no voters:
I20260928 23:07:00.360988 21291 raft_consensus.cc:2781] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:00.361155 21291 raft_consensus.cc:686] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 1 LEADER]: Becoming Leader. State: Replica: 1ba480bcb6824dcd8babe79cd58a4614, State: Running, Role: LEADER
I20260928 23:07:00.361249 21291 consensus_queue.cc:237] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } }
I20260928 23:07:00.362152 20824 catalog_manager.cc:5495] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ba480bcb6824dcd8babe79cd58a4614 (127.15.212.65). New cstate: current_term: 1 leader_uuid: "1ba480bcb6824dcd8babe79cd58a4614" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba480bcb6824dcd8babe79cd58a4614" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 41523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 } health_report { overall_health: UNKNOWN } } }
I20260928 23:07:00.381027 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.380076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.15.212.65:41523)
0928 23:07:00.380124 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.380125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.380139 (+ 14us) server_negotiation.cc:407] Connection header received
0928 23:07:00.380221 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.380222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.380233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.380279 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.380810 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.380811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.380811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.380815 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.380830 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.380833 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.380879 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.380919 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.380931 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:07:00.382751 21097 raft_consensus.cc:1261] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 1 FOLLOWER]: Refusing update from remote peer 1ba480bcb6824dcd8babe79cd58a4614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:00.382761 21228 raft_consensus.cc:1261] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 1 FOLLOWER]: Refusing update from remote peer 1ba480bcb6824dcd8babe79cd58a4614: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:00.383102 21291 consensus_queue.cc:1035] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dc9dcb89d154556aef028353359f7ec" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:00.383219 21298 consensus_queue.cc:1035] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b39330aa0284d959ffe177d9e1fd282" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:00.385842 21303 mvcc.cc:205] Tried to move back new op lower bound from 7334448416285057024 to 7334448416200466432. Current Snapshot: MvccSnapshot[applied={T|T < 7334448416285057024}]
I20260928 23:07:00.387454 20824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:00.387531 20824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:00.388180 20824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 260d6e1614a04e17b90c0b377cd811ab: Sending DeleteTablet for 3 replicas of tablet 6e177e488997452982af9ce6d9aea196
I20260928 23:07:00.388535 20946 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e177e488997452982af9ce6d9aea196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:00 UTC) from {username='slave'} at 127.0.0.1:55774
I20260928 23:07:00.388513 21077 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e177e488997452982af9ce6d9aea196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:00 UTC) from {username='slave'} at 127.0.0.1:60326
I20260928 23:07:00.388777 21310 tablet_replica.cc:330] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614: stopping tablet replica
I20260928 23:07:00.388760 21208 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e177e488997452982af9ce6d9aea196 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:00 UTC) from {username='slave'} at 127.0.0.1:48550
I20260928 23:07:00.388875 21310 raft_consensus.cc:2229] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:00.388965 21310 raft_consensus.cc:2258] T 6e177e488997452982af9ce6d9aea196 P 1ba480bcb6824dcd8babe79cd58a4614 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:00.389035 21312 tablet_replica.cc:330] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: stopping tablet replica
I20260928 23:07:00.389103 21311 tablet_replica.cc:330] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: stopping tablet replica
I20260928 23:07:00.389202 21311 raft_consensus.cc:2229] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:00.389273 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20884
I20260928 23:07:00.389307 21311 raft_consensus.cc:2258] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:00.389214 21312 raft_consensus.cc:2229] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:00.389500 21312 raft_consensus.cc:2258] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:00.390172 21311 ts_tablet_manager.cc:1905] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:00.391168 21312 ts_tablet_manager.cc:1905] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:00.392243 21311 ts_tablet_manager.cc:1918] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:00.392318 21311 log.cc:1196] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/6e177e488997452982af9ce6d9aea196
I20260928 23:07:00.392393 21312 ts_tablet_manager.cc:1918] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:00.392488 21312 log.cc:1196] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/6e177e488997452982af9ce6d9aea196
I20260928 23:07:00.392616 21311 ts_tablet_manager.cc:1939] T 6e177e488997452982af9ce6d9aea196 P 7dc9dcb89d154556aef028353359f7ec: Deleting consensus metadata
I20260928 23:07:00.392786 21312 ts_tablet_manager.cc:1939] T 6e177e488997452982af9ce6d9aea196 P 1b39330aa0284d959ffe177d9e1fd282: Deleting consensus metadata
I20260928 23:07:00.393075 20808 catalog_manager.cc:4841] TS 7dc9dcb89d154556aef028353359f7ec (127.15.212.66:43395): tablet 6e177e488997452982af9ce6d9aea196 (table test-table [id=1f4b8af08e7942ec95da94efffd82542]) successfully deleted
I20260928 23:07:00.393301 20811 catalog_manager.cc:4841] TS 1b39330aa0284d959ffe177d9e1fd282 (127.15.212.67:32957): tablet 6e177e488997452982af9ce6d9aea196 (table test-table [id=1f4b8af08e7942ec95da94efffd82542]) successfully deleted
W20260928 23:07:00.396299 20809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:41523 (error 108)
I20260928 23:07:00.396502 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21015
I20260928 23:07:00.397240 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.396663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:41523 (local address 127.0.0.1:55778)
0928 23:07:00.397162 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.397200 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:41523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479}
W20260928 23:07:00.397377 20809 catalog_manager.cc:4584] TS 1ba480bcb6824dcd8babe79cd58a4614 (127.15.212.65:41523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e177e488997452982af9ce6d9aea196: Network error: Client connection negotiation failed: client connection to 127.15.212.65:41523: connect: Connection refused (error 111)
I20260928 23:07:00.401192 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21146
I20260928 23:07:00.405861 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 20793
2026-09-28T23:07:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (757 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 23:07:00.424791 16209 mini_oidc.cc:152] Starting JWKS server
I20260928 23:07:00.425815 16209 webserver.cc:460] Webserver started at https://127.0.0.1:42097/ using document root <none> and password file <none>
I20260928 23:07:00.425930 16209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 23:07:00.426096 16209 webserver.cc:460] Webserver started at http://127.0.0.1:38615/ using document root <none> and password file <none>
2026-09-28T23:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:00Z Disabled control of system clock
I20260928 23:07:00.433642 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45771
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45771
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42097/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/testchainca.cert with env {}
W20260928 23:07:00.511229 21319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:00.511412 21319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:00.511435 21319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:00.512544 21319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:00.512581 21319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:00.512594 21319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:00.512607 21319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:00.512619 21319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 23:07:00.512636 21319 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 23:07:00.512647 21319 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42097/jwks.json
I20260928 23:07:00.514111 21319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42097/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:00.514281 21319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:00.514425 21319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:00.518970 21325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.518961 21327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.518963 21324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.518989 21319 server_base.cc:1034] running on GCE node
I20260928 23:07:00.519351 21319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:00.519600 21319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:00.520732 21319 hybrid_clock.cc:648] HybridClock initialized: now 1790636820520718 us; error 28 us; skew 500 ppm
I20260928 23:07:00.521811 21319 webserver.cc:460] Webserver started at http://127.15.212.126:41035/ using document root <none> and password file <none>
I20260928 23:07:00.522040 21319 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:00.522085 21319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:00.522217 21319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:00.523181 21319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "7fde0f36c68047c2951ce4212521e6b9"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "75ce16d2dd57d28340854133a26be290"
server_key_iv: "b6f4e9b1c7220cfa7f3c228b68fe997e"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.523506 21319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "7fde0f36c68047c2951ce4212521e6b9"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "75ce16d2dd57d28340854133a26be290"
server_key_iv: "b6f4e9b1c7220cfa7f3c228b68fe997e"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.524708 21319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:00.525343 21333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.525560 21319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:00.525658 21319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "7fde0f36c68047c2951ce4212521e6b9"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "75ce16d2dd57d28340854133a26be290"
server_key_iv: "b6f4e9b1c7220cfa7f3c228b68fe997e"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.525733 21319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:00.538127 21319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:00.538378 21319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:00.538511 21319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:00.542023 21319 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:45771
I20260928 23:07:00.542050 21385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:45771 every 8 connection(s)
I20260928 23:07:00.542346 21319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:00.543210 21386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.545243 21386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: Bootstrap starting.
I20260928 23:07:00.545761 21386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.546049 21386 log.cc:824] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.546681 21386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: No bootstrap required, opened a new log
I20260928 23:07:00.547365 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21319
I20260928 23:07:00.547451 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:00.547628 16209 external_mini_cluster.cc:1428] Setting key 5fe43cf8f77df8a96aaf6b198841c8ba
I20260928 23:07:00.548702 21386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } }
I20260928 23:07:00.548854 21386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.548899 21386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fde0f36c68047c2951ce4212521e6b9, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.549009 21386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } }
I20260928 23:07:00.549082 21386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:00.549111 21386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:00.549170 21386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.549837 21386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } }
I20260928 23:07:00.549937 21386 leader_election.cc:302] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fde0f36c68047c2951ce4212521e6b9; no voters:
I20260928 23:07:00.550073 21386 leader_election.cc:288] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:00.550148 21391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:00.550283 21391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [term 1 LEADER]: Becoming Leader. State: Replica: 7fde0f36c68047c2951ce4212521e6b9, State: Running, Role: LEADER
I20260928 23:07:00.550294 21386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:00.550379 21391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } }
I20260928 23:07:00.550675 21391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fde0f36c68047c2951ce4212521e6b9. Latest consensus state: current_term: 1 leader_uuid: "7fde0f36c68047c2951ce4212521e6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } } }
I20260928 23:07:00.550766 21391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:00.550684 21392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fde0f36c68047c2951ce4212521e6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fde0f36c68047c2951ce4212521e6b9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45771 } } }
I20260928 23:07:00.550853 21392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9 [sys.catalog]: This master's current role is: LEADER
W20260928 23:07:00.552037 21407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:00.552093 21407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:00.552147 21398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:00.552299 21398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:00.553679 21398 catalog_manager.cc:1347] Generated new cluster ID: 9bb55049ad594c98b3123ecf48e0f35e
I20260928 23:07:00.553728 21398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:00.554039 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.548074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.15.212.126:45771)
0928 23:07:00.548297 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.548305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.548335 (+ 30us) server_negotiation.cc:407] Connection header received
0928 23:07:00.548662 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.548669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.548742 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.548881 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.549071 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.550819 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.552005 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.552608 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.552737 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.553351 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.553354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.553355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.553523 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.553552 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.553561 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.553652 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.553756 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.553830 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260928 23:07:00.557179 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:07:00.560837 21398 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:00.561400 21398 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:00.568657 21398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fde0f36c68047c2951ce4212521e6b9: Generated new TSK 0
I20260928 23:07:00.568814 21398 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 23:07:00.593838 21416 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 23:07:00.593923 21416 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 23:07:00.594136 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.579356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.15.212.126:45771)
0928 23:07:00.579400 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.579402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.579506 (+ 104us) server_negotiation.cc:407] Connection header received
0928 23:07:00.579591 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.579591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.579600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.579656 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0928 23:07:00.579813 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.580282 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.581390 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.581562 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.581614 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.581846 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 23:07:00.593952 (+ 12106us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 23:07:00.594088 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40654: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 23:07:00.594195 21390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40654: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 23:07:00.594516 21415 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:00.579247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:45771 (local address 127.0.0.1:40654)
0928 23:07:00.579412 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.579422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.579506 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.579676 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.579677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.579679 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 23:07:00.579747 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.579749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.580346 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.580347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.581311 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.581314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.581645 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.581647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.581716 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.581736 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 23:07:00.594093 (+ 12357us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 23:07:00.594209 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.212.126:45771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.212.126:45771: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 23:07:00.594976 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21319
2026-09-28T23:07:00Z chronyd exiting
I20260928 23:07:00.630831 16209 test_util.cc:175] -----------------------------------------------
I20260928 23:07:00.630930 16209 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790636798761935-16209-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (207 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 23:07:00.631737 16209 mini_oidc.cc:152] Starting JWKS server
I20260928 23:07:00.632517 16209 webserver.cc:460] Webserver started at https://127.0.0.1:45043/ using document root <none> and password file <none>
I20260928 23:07:00.632584 16209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 23:07:00.632763 16209 webserver.cc:460] Webserver started at http://127.0.0.1:38337/ using document root <none> and password file <none>
2026-09-28T23:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:00Z Disabled control of system clock
I20260928 23:07:00.640496 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40085
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:40085
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45043/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/testchainca.cert with env {}
W20260928 23:07:00.716871 21422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:00.717062 21422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:00.717083 21422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:00.718191 21422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:00.718226 21422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:00.718240 21422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:00.718251 21422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:00.718263 21422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 23:07:00.718276 21422 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 23:07:00.718293 21422 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45043/jwks.json
I20260928 23:07:00.719693 21422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:40085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45043/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:00.719869 21422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:00.720010 21422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:00.724581 21427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.724557 21428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.724565 21430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.724771 21422 server_base.cc:1034] running on GCE node
I20260928 23:07:00.725049 21422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:00.725356 21422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:00.726503 21422 hybrid_clock.cc:648] HybridClock initialized: now 1790636820726491 us; error 29 us; skew 500 ppm
I20260928 23:07:00.727643 21422 webserver.cc:460] Webserver started at http://127.15.212.126:37747/ using document root <none> and password file <none>
I20260928 23:07:00.727814 21422 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:00.727850 21422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:00.727928 21422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:00.728865 21422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "3f85ea8d9d134fa6b27bf5f108a479b0"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "5747c466922182afeee0391b9e6e4cf6"
server_key_iv: "c6f75def3e3e500463aec3eb0a238059"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.729135 21422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "3f85ea8d9d134fa6b27bf5f108a479b0"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "5747c466922182afeee0391b9e6e4cf6"
server_key_iv: "c6f75def3e3e500463aec3eb0a238059"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.730268 21422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:00.730916 21436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.731112 21422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:00.731197 21422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "3f85ea8d9d134fa6b27bf5f108a479b0"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "5747c466922182afeee0391b9e6e4cf6"
server_key_iv: "c6f75def3e3e500463aec3eb0a238059"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.731247 21422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:00.743240 21422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:00.743455 21422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:00.743556 21422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:00.747068 21422 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:40085
I20260928 23:07:00.747151 21488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:40085 every 8 connection(s)
I20260928 23:07:00.747428 21422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:00.748051 21489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.750617 21489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: Bootstrap starting.
I20260928 23:07:00.751168 21489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.751433 21489 log.cc:824] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.752067 21489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: No bootstrap required, opened a new log
I20260928 23:07:00.753590 21489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } }
I20260928 23:07:00.753727 21489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.753749 21489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f85ea8d9d134fa6b27bf5f108a479b0, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.753918 21489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } }
I20260928 23:07:00.753993 21489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:00.754037 21489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:00.754086 21489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.754307 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21422
I20260928 23:07:00.754407 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:00.754568 16209 external_mini_cluster.cc:1428] Setting key 7d6dee4cb80ba885c4ca1331b44466dc
I20260928 23:07:00.754666 21489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } }
I20260928 23:07:00.754779 21489 leader_election.cc:302] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f85ea8d9d134fa6b27bf5f108a479b0; no voters:
I20260928 23:07:00.754927 21489 leader_election.cc:288] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:00.754990 21492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:00.755168 21492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [term 1 LEADER]: Becoming Leader. State: Replica: 3f85ea8d9d134fa6b27bf5f108a479b0, State: Running, Role: LEADER
I20260928 23:07:00.755183 21489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:00.755265 21492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } }
I20260928 23:07:00.755569 21492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } } }
I20260928 23:07:00.755642 21492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:00.755777 21492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f85ea8d9d134fa6b27bf5f108a479b0. Latest consensus state: current_term: 1 leader_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f85ea8d9d134fa6b27bf5f108a479b0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40085 } } }
I20260928 23:07:00.755839 21492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0 [sys.catalog]: This master's current role is: LEADER
W20260928 23:07:00.757232 21510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:00.757287 21510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:00.757333 21501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:00.757480 21501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:00.760006 21501 catalog_manager.cc:1347] Generated new cluster ID: 25ce988ae517467f925222e65d26ee72
I20260928 23:07:00.760071 21501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:00.761088 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.754993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.15.212.126:40085)
0928 23:07:00.756087 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.756097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.756131 (+ 34us) server_negotiation.cc:407] Connection header received
0928 23:07:00.756254 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.756264 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.756343 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.756543 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.756889 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.757980 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.759087 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.759483 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.759580 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.760314 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.760319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.760319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.760548 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.760588 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.760602 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.760654 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.760794 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.760868 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1021,"thread_start_us":64,"threads_started":1}
I20260928 23:07:00.764830 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:07:00.769959 21501 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:00.770601 21501 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:00.773732 21501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f85ea8d9d134fa6b27bf5f108a479b0: Generated new TSK 0
I20260928 23:07:00.773871 21501 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 23:07:00.795058 21519 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 23:07:00.795143 21519 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 23:07:00.795449 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.781185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42928 (local address 127.15.212.126:40085)
0928 23:07:00.781221 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.781224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.781330 (+ 106us) server_negotiation.cc:407] Connection header received
0928 23:07:00.781395 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.781396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.781416 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.781517 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0928 23:07:00.781710 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.782448 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.783212 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.783525 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.783587 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.783728 (+ 141us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 23:07:00.795228 (+ 11500us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 23:07:00.795369 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42928: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 23:07:00.795508 21495 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42928: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 23:07:00.795523 21518 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:00.781139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40085 (local address 127.0.0.1:42928)
0928 23:07:00.781265 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.781275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.781334 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.781540 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.781541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.781543 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 23:07:00.781610 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.781612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.782552 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.782553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.783141 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.783143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.783601 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.783602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.783646 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.783651 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 23:07:00.795346 (+ 11695us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 23:07:00.795427 (+ 81us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.212.126:40085: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45043/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":88,"threads_started":1}
I20260928 23:07:00.795835 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21422
2026-09-28T23:07:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (201 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 23:07:00.833065 16209 mini_oidc.cc:152] Starting JWKS server
I20260928 23:07:00.833730 16209 webserver.cc:460] Webserver started at https://127.0.0.1:37477/ using document root <none> and password file <none>
I20260928 23:07:00.833771 16209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 23:07:00.833945 16209 webserver.cc:460] Webserver started at http://127.0.0.1:32909/ using document root <none> and password file <none>
2026-09-28T23:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:00Z Disabled control of system clock
I20260928 23:07:00.841287 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:33759
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:33759
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37477/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 23:07:00.918206 21525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:00.918380 21525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:00.918401 21525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:00.919452 21525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:00.919484 21525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:00.919497 21525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:00.919510 21525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:00.919523 21525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 23:07:00.919534 21525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 23:07:00.919545 21525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37477/jwks.json
I20260928 23:07:00.920850 21525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:33759
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:33759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37477/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:00.921010 21525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:00.921221 21525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:00.925659 21533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:00.925647 21530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.925827 21525 server_base.cc:1034] running on GCE node
W20260928 23:07:00.925661 21531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:00.926088 21525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:00.926345 21525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:00.927525 21525 hybrid_clock.cc:648] HybridClock initialized: now 1790636820927511 us; error 38 us; skew 500 ppm
I20260928 23:07:00.928730 21525 webserver.cc:460] Webserver started at http://127.15.212.126:33639/ using document root <none> and password file <none>
I20260928 23:07:00.928957 21525 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:00.929021 21525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:00.929176 21525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:00.930150 21525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "9f220f6020f7a4f696733cc9db00acc9"
server_key_iv: "e396d99cda579e59e7eac3ad97d9c76b"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.930482 21525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "9f220f6020f7a4f696733cc9db00acc9"
server_key_iv: "e396d99cda579e59e7eac3ad97d9c76b"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.931685 21525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:00.932365 21539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:00.932542 21525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:00.932636 21525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02"
format_stamp: "Formatted at 2026-09-28 23:07:00 on dist-test-slave-x2t8"
server_key: "9f220f6020f7a4f696733cc9db00acc9"
server_key_iv: "e396d99cda579e59e7eac3ad97d9c76b"
server_key_version: "encryptionkey@0"
I20260928 23:07:00.932709 21525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:00.940835 21525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:00.941104 21525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:00.941243 21525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:00.944796 21525 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:33759
I20260928 23:07:00.944836 21591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:33759 every 8 connection(s)
I20260928 23:07:00.945186 21525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:00.945806 21592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:00.948284 21592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02: Bootstrap starting.
I20260928 23:07:00.948853 21592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:00.949122 21592 log.cc:824] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:00.949770 21592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02: No bootstrap required, opened a new log
I20260928 23:07:00.951269 21592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } }
I20260928 23:07:00.951385 21592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:00.951406 21592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab4694e10464ffa9f4d1ef84f4f6c02, State: Initialized, Role: FOLLOWER
I20260928 23:07:00.951565 21592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } }
I20260928 23:07:00.951622 21592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:00.951658 21592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:00.951704 21592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:00.952229 21592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } }
I20260928 23:07:00.952328 21592 leader_election.cc:302] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ab4694e10464ffa9f4d1ef84f4f6c02; no voters:
I20260928 23:07:00.952531 21592 leader_election.cc:288] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:00.952585 21595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:00.952726 21595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [term 1 LEADER]: Becoming Leader. State: Replica: 7ab4694e10464ffa9f4d1ef84f4f6c02, State: Running, Role: LEADER
I20260928 23:07:00.952767 21592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:00.952821 21595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } }
I20260928 23:07:00.953166 21596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } } }
I20260928 23:07:00.953302 21596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:00.953327 21597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ab4694e10464ffa9f4d1ef84f4f6c02. Latest consensus state: current_term: 1 leader_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab4694e10464ffa9f4d1ef84f4f6c02" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 33759 } } }
I20260928 23:07:00.953423 21597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:00.953781 21604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:00.954275 21604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:00.954885 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21525
I20260928 23:07:00.954983 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:00.955137 16209 external_mini_cluster.cc:1428] Setting key b508254a0add8edcbc5916e3f12a86e3
I20260928 23:07:00.957028 21604 catalog_manager.cc:1347] Generated new cluster ID: 813a21b475844a4a97fd37927040de49
I20260928 23:07:00.957084 21604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:00.961911 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.955514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.15.212.126:33759)
0928 23:07:00.955703 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.955711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.956644 (+ 933us) server_negotiation.cc:407] Connection header received
0928 23:07:00.956793 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.956800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.956866 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.956995 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:00.957307 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.958721 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.959764 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.960383 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.960525 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.961163 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:00.961167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:00.961167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:00.961364 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:00.961409 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:00.961425 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:00.961506 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:00.961579 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 23:07:00.961678 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260928 23:07:00.962797 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:07:00.976878 21604 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:00.977348 21604 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:00.983479 21604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ab4694e10464ffa9f4d1ef84f4f6c02: Generated new TSK 0
I20260928 23:07:00.983657 21604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:00.996098 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:00.992617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.15.212.126:33759)
0928 23:07:00.992671 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:00.992675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:00.992791 (+ 116us) server_negotiation.cc:407] Connection header received
0928 23:07:00.992868 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:00.992869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:00.992878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:00.992934 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0928 23:07:00.993049 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.993523 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.994667 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.994852 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.994898 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.995065 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 23:07:00.995904 (+ 839us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 23:07:00.996027 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39958: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":32}
W20260928 23:07:00.996200 21621 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:00.992562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:33759 (local address 127.0.0.1:39958)
0928 23:07:00.992726 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:00.992738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:00.992803 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:00.992936 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:00.992936 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:00.992938 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 23:07:00.992992 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.992994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.993593 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.993593 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.994574 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:00.994577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:00.994920 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:00.994921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:00.994977 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:00.994983 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 23:07:00.995996 (+ 1013us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 23:07:00.996094 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.212.126:33759: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37477/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":117,"threads_started":1}
I20260928 23:07:00.996397 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21525
E20260928 23:07:01.006824 21622 webserver.cc:533] Webserver: sslize failed:
E20260928 23:07:01.006877 21622 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-28T23:07:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (200 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T23:07:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:01Z Disabled control of system clock
I20260928 23:07:01.039944 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37807
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37807
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/testchainca.cert with env {}
W20260928 23:07:01.119011 21626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:01.119190 21626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:01.119210 21626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:01.120294 21626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:01.120327 21626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:01.120339 21626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:01.120352 21626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:01.120364 21626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 23:07:01.120376 21626 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 23:07:01.120388 21626 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 23:07:01.121840 21626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:01.122015 21626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:01.122157 21626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:01.126796 21634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:01.126783 21631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:01.126891 21626 server_base.cc:1034] running on GCE node
W20260928 23:07:01.126775 21632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:01.127219 21626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:01.127483 21626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:01.128618 21626 hybrid_clock.cc:648] HybridClock initialized: now 1790636821128601 us; error 34 us; skew 500 ppm
I20260928 23:07:01.129814 21626 webserver.cc:460] Webserver started at http://127.15.212.126:45419/ using document root <none> and password file <none>
I20260928 23:07:01.130030 21626 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:01.130074 21626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:01.130209 21626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:01.131217 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "0b1e1ee625c64928a8008f21e3b42083"
format_stamp: "Formatted at 2026-09-28 23:07:01 on dist-test-slave-x2t8"
server_key: "d8369b7135c66aaae7ea19bb30435157"
server_key_iv: "ee48cf38678ba0ed27b718bc7483e0b0"
server_key_version: "encryptionkey@0"
I20260928 23:07:01.131546 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "0b1e1ee625c64928a8008f21e3b42083"
format_stamp: "Formatted at 2026-09-28 23:07:01 on dist-test-slave-x2t8"
server_key: "d8369b7135c66aaae7ea19bb30435157"
server_key_iv: "ee48cf38678ba0ed27b718bc7483e0b0"
server_key_version: "encryptionkey@0"
I20260928 23:07:01.132790 21626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:01.133468 21640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:01.133679 21626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:01.133771 21626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "0b1e1ee625c64928a8008f21e3b42083"
format_stamp: "Formatted at 2026-09-28 23:07:01 on dist-test-slave-x2t8"
server_key: "d8369b7135c66aaae7ea19bb30435157"
server_key_iv: "ee48cf38678ba0ed27b718bc7483e0b0"
server_key_version: "encryptionkey@0"
I20260928 23:07:01.133846 21626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:01.145884 21626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:01.146127 21626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:01.146286 21626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:01.149863 21626 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:37807
I20260928 23:07:01.149916 21692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:37807 every 8 connection(s)
I20260928 23:07:01.150221 21626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:01.151106 21693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:01.153282 21693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083: Bootstrap starting.
I20260928 23:07:01.153836 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21626
I20260928 23:07:01.153910 21693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:01.153964 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:01.154171 16209 external_mini_cluster.cc:1428] Setting key f21cb15b1fec4080cdc033911a697b7d
I20260928 23:07:01.154222 21693 log.cc:824] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:01.155074 21693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083: No bootstrap required, opened a new log
I20260928 23:07:01.157631 21693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } }
I20260928 23:07:01.157770 21693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:01.157791 21693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1e1ee625c64928a8008f21e3b42083, State: Initialized, Role: FOLLOWER
I20260928 23:07:01.157920 21693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } }
I20260928 23:07:01.157987 21693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:01.158026 21693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:01.158073 21693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:01.158663 21693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } }
I20260928 23:07:01.158768 21693 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b1e1ee625c64928a8008f21e3b42083; no voters:
I20260928 23:07:01.158916 21693 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:01.158972 21698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:01.159102 21698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1e1ee625c64928a8008f21e3b42083, State: Running, Role: LEADER
I20260928 23:07:01.159155 21693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:01.159204 21698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } }
I20260928 23:07:01.159482 21698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b1e1ee625c64928a8008f21e3b42083. Latest consensus state: current_term: 1 leader_uuid: "0b1e1ee625c64928a8008f21e3b42083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } } }
I20260928 23:07:01.159566 21698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:01.159493 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b1e1ee625c64928a8008f21e3b42083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1e1ee625c64928a8008f21e3b42083" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37807 } } }
I20260928 23:07:01.159669 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:01.159755 21702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:01.160207 21702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:01.160619 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:01.154628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.15.212.126:37807)
0928 23:07:01.154828 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:01.154837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:01.154867 (+ 30us) server_negotiation.cc:407] Connection header received
0928 23:07:01.155455 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:01.155464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:01.155536 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:01.155699 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:01.155936 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.157451 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.158402 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.159029 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.159479 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:01.159943 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:01.159946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:01.159947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:01.160143 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:01.160182 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:01.160196 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:01.160265 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:01.160339 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 23:07:01.160414 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260928 23:07:01.161693 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:07:01.163170 21702 catalog_manager.cc:1347] Generated new cluster ID: 2d0796591ec44396b6694125ad384518
I20260928 23:07:01.163219 21702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:01.173548 21702 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:01.174017 21702 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:01.176186 21702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b1e1ee625c64928a8008f21e3b42083: Generated new TSK 0
I20260928 23:07:01.176354 21702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:01.273219 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:01.183550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.15.212.126:37807)
0928 23:07:01.183607 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:01.183610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:01.183713 (+ 103us) server_negotiation.cc:407] Connection header received
0928 23:07:01.183790 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:01.183791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:01.183803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:01.183868 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0928 23:07:01.184005 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.184533 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.185340 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.185558 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.185631 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:01.185822 (+ 191us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 23:07:01.272933 (+ 87111us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 23:07:01.273112 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44946: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 23:07:01.273296 21697 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44946: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 23:07:01.273300 21722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:01.183492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37807 (local address 127.0.0.1:44946)
0928 23:07:01.183656 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:01.183665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:01.183729 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:01.183877 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:01.183878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:01.183880 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 23:07:01.183936 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:01.183939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.184638 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.184639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:01.185244 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:01.185247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:01.185646 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:01.185647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:01.185714 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:01.185719 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 23:07:01.273070 (+ 87351us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 23:07:01.273198 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.212.126:37807: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260928 23:07:01.273756 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21626
2026-09-28T23:07:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (257 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:01 dist-test-slave-x2t8 krb5kdc[21734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:01 dist-test-slave-x2t8 krb5kdc[21734](info): set up 2 sockets
Sep 28 23:07:01 dist-test-slave-x2t8 krb5kdc[21734](info): commencing operation
krb5kdc: starting...
W20260928 23:07:02.476492 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:07:02.585376 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36337
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:36337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:02.663046 21747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:02.663261 21747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:02.663295 21747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:02.664390 21747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:02.664469 21747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:02.664495 21747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:02.664535 21747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:02.664559 21747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:02.665930 21747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:36337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:02.666173 21747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:02.666381 21747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:02.670843 21753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:02.670852 21755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:02.670852 21752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:02.671096 21747 server_base.cc:1034] running on GCE node
I20260928 23:07:02.671339 21747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:02.671588 21747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:02.672735 21747 hybrid_clock.cc:648] HybridClock initialized: now 1790636822672712 us; error 38 us; skew 500 ppm
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:02.674914 21747 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:07:02.675325 21747 webserver.cc:460] Webserver started at http://127.15.212.126:39489/ using document root <none> and password file <none>
I20260928 23:07:02.675490 21747 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:02.675529 21747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:02.675609 21747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:02.676569 21747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "04c48c8402c04f5ebe2394161a180714"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "fcd08f4595e1f09ff2f592b6b2f1d57d"
server_key_iv: "fa80160e7eed5f30123318ba930d9041"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.676863 21747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "04c48c8402c04f5ebe2394161a180714"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "fcd08f4595e1f09ff2f592b6b2f1d57d"
server_key_iv: "fa80160e7eed5f30123318ba930d9041"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.678014 21747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:02.678599 21762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:02.678779 21747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:02.678886 21747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "04c48c8402c04f5ebe2394161a180714"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "fcd08f4595e1f09ff2f592b6b2f1d57d"
server_key_iv: "fa80160e7eed5f30123318ba930d9041"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.678954 21747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:02.697629 21747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:02.699723 21747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:02.699857 21747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:02.703414 21747 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:36337
I20260928 23:07:02.703459 21814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:36337 every 8 connection(s)
I20260928 23:07:02.703749 21747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:02.704627 21815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:02.706655 21815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714: Bootstrap starting.
I20260928 23:07:02.707149 21815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:02.707378 21815 log.cc:824] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:02.707974 21815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714: No bootstrap required, opened a new log
I20260928 23:07:02.709409 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21747
I20260928 23:07:02.709523 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:02.709707 16209 external_mini_cluster.cc:1428] Setting key d6faa56fbfcbdab5d8dfb89c98dbff57
I20260928 23:07:02.709745 21815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } }
I20260928 23:07:02.709929 21815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:02.709977 21815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c48c8402c04f5ebe2394161a180714, State: Initialized, Role: FOLLOWER
I20260928 23:07:02.710124 21815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } }
I20260928 23:07:02.710225 21815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:02.710311 21815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:02.710379 21815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:02.711510 21815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } }
I20260928 23:07:02.711596 21815 leader_election.cc:302] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04c48c8402c04f5ebe2394161a180714; no voters:
I20260928 23:07:02.711766 21815 leader_election.cc:288] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:02.711795 21820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:02.711905 21820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [term 1 LEADER]: Becoming Leader. State: Replica: 04c48c8402c04f5ebe2394161a180714, State: Running, Role: LEADER
I20260928 23:07:02.712030 21815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:02.712040 21820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } }
I20260928 23:07:02.712505 21822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04c48c8402c04f5ebe2394161a180714. Latest consensus state: current_term: 1 leader_uuid: "04c48c8402c04f5ebe2394161a180714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } } }
I20260928 23:07:02.712596 21822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:02.713018 21821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04c48c8402c04f5ebe2394161a180714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c48c8402c04f5ebe2394161a180714" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36337 } } }
I20260928 23:07:02.713095 21821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:02.713238 21827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:02.713647 21827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:02.715459 21827 catalog_manager.cc:1347] Generated new cluster ID: b6a41ff4e98044a188ad4c31ffda5031
I20260928 23:07:02.715516 21827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:02.720723 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:02.710139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.15.212.126:36337)
0928 23:07:02.710316 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:02.710325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:02.710373 (+ 48us) server_negotiation.cc:407] Connection header received
0928 23:07:02.710492 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:02.710499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:02.710543 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:02.710653 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:02.711442 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.712716 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.713824 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.714435 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.714545 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:02.717258 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:02.717263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:02.717263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:02.717273 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:02.719105 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:02.719522 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:02.719525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:02.719525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:02.719642 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:02.719852 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:02.719853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:02.719853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:02.720117 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:02.720252 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:02.720367 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 23:07:02.720508 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260928 23:07:02.723024 21827 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:02.723464 21827 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:02.726004 21827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04c48c8402c04f5ebe2394161a180714: Generated new TSK 0
I20260928 23:07:02.726158 21827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:07:02.757226 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:02.835240 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:02.835448 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:02.835482 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:02.836643 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:02.836755 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:02.836855 21841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:02.838521 21841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:02.838774 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:02.838984 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:02.843709 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:02.843732 21849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:02.843715 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:02.843974 21841 server_base.cc:1034] running on GCE node
I20260928 23:07:02.844256 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:02.844558 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:02.845733 21841 hybrid_clock.cc:648] HybridClock initialized: now 1790636822845723 us; error 45 us; skew 500 ppm
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:02.847993 21841 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:07:02.848397 21841 webserver.cc:460] Webserver started at http://127.15.212.65:44953/ using document root <none> and password file <none>
I20260928 23:07:02.848629 21841 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:02.848693 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:02.848829 21841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:02.849782 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "5bc8ed6e3bcc4966a93cbe149ec70836"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "49160438f7523db1d92b26f28898e5bd"
server_key_iv: "8c0bbfc6ce925b6d5baf3e32d6e8c6f7"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.850131 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "5bc8ed6e3bcc4966a93cbe149ec70836"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "49160438f7523db1d92b26f28898e5bd"
server_key_iv: "8c0bbfc6ce925b6d5baf3e32d6e8c6f7"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.851393 21841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:02.852028 21856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:02.852187 21841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:02.852276 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "5bc8ed6e3bcc4966a93cbe149ec70836"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "49160438f7523db1d92b26f28898e5bd"
server_key_iv: "8c0bbfc6ce925b6d5baf3e32d6e8c6f7"
server_key_version: "encryptionkey@0"
I20260928 23:07:02.852355 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:02.865756 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:02.867336 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:02.867482 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:02.867684 21841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:02.867961 21841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:02.868017 21841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:02.868057 21841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:02.868085 21841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:02.873849 21841 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:42919
I20260928 23:07:02.873894 21969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:42919 every 8 connection(s)
I20260928 23:07:02.874192 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:02.881609 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21841
I20260928 23:07:02.881724 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:02.881937 16209 external_mini_cluster.cc:1428] Setting key 633c2e12dd78179bf3010cd8a2b2cf97
I20260928 23:07:02.883416 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:02.874866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:36433 (local address 127.15.212.126:36337)
0928 23:07:02.874929 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:02.874931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:02.875154 (+ 223us) server_negotiation.cc:407] Connection header received
0928 23:07:02.876027 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:02.876028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:02.876040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:02.876095 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:02.876710 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.877206 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.878166 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.878350 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.878399 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:02.880317 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:02.880318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:02.880318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:02.880324 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:02.881987 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:02.882459 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:02.882460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:02.882461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:02.882522 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:02.882823 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:02.882824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:02.882824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:02.882929 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:02.882978 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:02.883295 (+ 317us) server_negotiation.cc:299] Negotiation successful
0928 23:07:02.883356 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:07:02.883697 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:02.874861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36337 (local address 127.15.212.65:36433)
0928 23:07:02.875024 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:02.875064 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:02.875938 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:02.876147 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:02.876150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:02.876382 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:02.876632 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:02.876637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.877281 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.877281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:02.878087 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:02.878090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:02.878418 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:02.878419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:02.878514 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:02.878935 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:02.878943 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:02.880232 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:02.882062 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:02.882069 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:02.882071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:02.882364 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:02.882597 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:02.882598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:02.882598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:02.882737 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:02.882989 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:02.882995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:02.883168 (+ 173us) client_negotiation.cc:769] Sending connection context
0928 23:07:02.883276 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 23:07:02.883439 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260928 23:07:02.884213 21970 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36337
I20260928 23:07:02.884297 21970 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:02.884513 21970 heartbeater.cc:511] Master 127.15.212.126:36337 requested a full tablet report, sending...
I20260928 23:07:02.885006 21779 ts_manager.cc:194] Registered new tserver with Master: 5bc8ed6e3bcc4966a93cbe149ec70836 (127.15.212.65:42919)
I20260928 23:07:02.885588 21779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:36433
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:07:02.907028 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:02.986899 21975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:02.987123 21975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:02.987162 21975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:02.988350 21975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:02.988415 21975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:02.988476 21975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:02.989921 21975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:02.990182 21975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:02.990409 21975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:02.995296 21983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:02.995364 21980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:02.995412 21975 server_base.cc:1034] running on GCE node
W20260928 23:07:02.995350 21981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:02.995736 21975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:02.995929 21975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:02.997061 21975 hybrid_clock.cc:648] HybridClock initialized: now 1790636822997045 us; error 35 us; skew 500 ppm
Sep 28 23:07:02 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:02.999428 21975 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:07:02.999858 21975 webserver.cc:460] Webserver started at http://127.15.212.66:46809/ using document root <none> and password file <none>
I20260928 23:07:03.000089 21975 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:03.000157 21975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:03.000299 21975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:03.001400 21975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "a6cd5fd7c93b41c9a8f427e4ea2b28e7"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "403a2f5c90a39fdf4ac2b8f1d461ac4e"
server_key_iv: "9497a7550f75cde07d217b0195e39a9a"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.001763 21975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "a6cd5fd7c93b41c9a8f427e4ea2b28e7"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "403a2f5c90a39fdf4ac2b8f1d461ac4e"
server_key_iv: "9497a7550f75cde07d217b0195e39a9a"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.003063 21975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:03.003791 21990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.003953 21975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:03.004046 21975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "a6cd5fd7c93b41c9a8f427e4ea2b28e7"
format_stamp: "Formatted at 2026-09-28 23:07:02 on dist-test-slave-x2t8"
server_key: "403a2f5c90a39fdf4ac2b8f1d461ac4e"
server_key_iv: "9497a7550f75cde07d217b0195e39a9a"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.004124 21975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:03.021142 21975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:03.023080 21975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:03.023244 21975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:03.023447 21975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:03.023746 21975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:03.023795 21975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.023836 21975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:03.023864 21975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.029537 21975 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:33383
I20260928 23:07:03.029615 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:33383 every 8 connection(s)
I20260928 23:07:03.029932 21975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:03.031499 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 21975
I20260928 23:07:03.031589 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:03.031764 16209 external_mini_cluster.cc:1428] Setting key 6a100576ba89b5f560e892dbfe4b8664
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:03.040217 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.030592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:51799 (local address 127.15.212.126:36337)
0928 23:07:03.030650 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.030652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.030870 (+ 218us) server_negotiation.cc:407] Connection header received
0928 23:07:03.031824 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.031825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.031837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.031888 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.032677 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.033174 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.034575 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.034884 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.034979 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.037341 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:03.037342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:03.037342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:03.037348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:03.038758 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.039218 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.039219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.039219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.039273 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.039536 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.039537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.039537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.039649 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:03.039716 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:03.040062 (+ 346us) server_negotiation.cc:299] Negotiation successful
0928 23:07:03.040124 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:07:03.040483 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.030597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36337 (local address 127.15.212.66:51799)
0928 23:07:03.030743 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:03.030776 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:03.031721 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:03.031966 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:03.031970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:03.032296 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:03.032594 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:03.032599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.033272 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.033273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:03.034481 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:03.034483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.034988 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.034989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:03.035106 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.035594 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:03.035603 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:03.037250 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:03.038834 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:03.038839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:03.038841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:03.039144 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:03.039338 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:03.039338 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:03.039339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:03.039472 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:03.039746 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:03.039752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:03.039922 (+ 170us) client_negotiation.cc:769] Sending connection context
0928 23:07:03.040052 (+ 130us) client_negotiation.cc:240] Negotiation successful
0928 23:07:03.040205 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260928 23:07:03.040917 22104 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36337
I20260928 23:07:03.041013 22104 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:03.041213 22104 heartbeater.cc:511] Master 127.15.212.126:36337 requested a full tablet report, sending...
I20260928 23:07:03.041606 21779 ts_manager.cc:194] Registered new tserver with Master: a6cd5fd7c93b41c9a8f427e4ea2b28e7 (127.15.212.66:33383)
I20260928 23:07:03.041970 21779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:51799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:07:03.057188 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:03.136624 22109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:03.136818 22109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:03.136842 22109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:03.138005 22109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:03.138044 22109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:03.138070 22109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:03.139479 22109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:03.139707 22109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:03.139935 22109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:03.144847 22115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:03.144855 22114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:03.144888 22109 server_base.cc:1034] running on GCE node
W20260928 23:07:03.144876 22117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:03.145249 22109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:03.145483 22109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:03.146648 22109 hybrid_clock.cc:648] HybridClock initialized: now 1790636823146635 us; error 33 us; skew 500 ppm
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.149073 22109 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:07:03.149514 22109 webserver.cc:460] Webserver started at http://127.15.212.67:37969/ using document root <none> and password file <none>
I20260928 23:07:03.149741 22109 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:03.149806 22109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:03.149946 22109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:03.150942 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "d317b4bf03124798910e9ca8f5d6fd3e"
format_stamp: "Formatted at 2026-09-28 23:07:03 on dist-test-slave-x2t8"
server_key: "e77f907ed86299150ab58ad00e0ffb06"
server_key_iv: "86df4802feb4ae0497679a3df80964b1"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.151292 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "d317b4bf03124798910e9ca8f5d6fd3e"
format_stamp: "Formatted at 2026-09-28 23:07:03 on dist-test-slave-x2t8"
server_key: "e77f907ed86299150ab58ad00e0ffb06"
server_key_iv: "86df4802feb4ae0497679a3df80964b1"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.152740 22109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:07:03.153442 22124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.153611 22109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:03.153703 22109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "d317b4bf03124798910e9ca8f5d6fd3e"
format_stamp: "Formatted at 2026-09-28 23:07:03 on dist-test-slave-x2t8"
server_key: "e77f907ed86299150ab58ad00e0ffb06"
server_key_iv: "86df4802feb4ae0497679a3df80964b1"
server_key_version: "encryptionkey@0"
I20260928 23:07:03.153776 22109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:03.166075 22109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:03.167936 22109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:03.168094 22109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:03.168304 22109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:03.168658 22109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:03.168707 22109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.168742 22109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:03.168768 22109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:03.174494 22109 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:39231
I20260928 23:07:03.174537 22237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:39231 every 8 connection(s)
I20260928 23:07:03.174836 22109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:03.181442 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22109
I20260928 23:07:03.181542 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:03.181744 16209 external_mini_cluster.cc:1428] Setting key cd55ba54f248b33f209fa0fa2425d12c
I20260928 23:07:03.184172 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.175514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57125 (local address 127.15.212.126:36337)
0928 23:07:03.175576 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.175578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.175773 (+ 195us) server_negotiation.cc:407] Connection header received
0928 23:07:03.176734 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.176736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.176748 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.176802 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.177423 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.177926 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.178906 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.179108 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.179188 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.181276 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:03.181278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:03.181278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:03.181286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:03.182856 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.183241 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.183242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.183243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.183300 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.183548 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.183549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.183550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.183667 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:03.183740 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:03.183996 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 23:07:03.184082 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:03.184360 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.175500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36337 (local address 127.15.212.67:57125)
0928 23:07:03.175659 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:03.175692 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:03.176620 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:03.176884 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:03.176887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:03.177139 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:03.177324 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:03.177328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.178018 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.178019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:03.178813 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:03.178815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.179192 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.179193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:03.179315 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.179791 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:03.179799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:03.181176 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:03.182927 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:03.182933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:03.182936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:03.183155 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:03.183368 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:03.183368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:03.183369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:03.183477 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:03.183730 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:03.183736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:03.183870 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 23:07:03.183973 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 23:07:03.184125 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260928 23:07:03.184774 22238 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36337
I20260928 23:07:03.184857 22238 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:03.184988 22238 heartbeater.cc:511] Master 127.15.212.126:36337 requested a full tablet report, sending...
I20260928 23:07:03.185295 21779 ts_manager.cc:194] Registered new tserver with Master: d317b4bf03124798910e9ca8f5d6fd3e (127.15.212.67:39231)
I20260928 23:07:03.185614 21779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:57125
I20260928 23:07:03.193475 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/test-admin.keytab.
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.208568 16209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 23:07:03.210121 22245 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:03.209378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.15.212.65:42919)
0928 23:07:03.209544 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.209549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.209558 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:03.209582 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.209584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.209597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.209660 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.210017 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49266: BlockingRecv error: recv got EOF from 127.0.0.1:49266 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
W20260928 23:07:03.210209 16209 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc' of type FILE
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.211185 16209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.212137 16209 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:07:03.219583 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.212728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.15.212.65:42919)
0928 23:07:03.212781 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.212784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.212820 (+ 36us) server_negotiation.cc:407] Connection header received
0928 23:07:03.212872 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.212873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.212882 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.212927 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.213300 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.214109 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.214781 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.215092 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.215150 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.217127 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:03.217134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:03.217134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:03.217140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:03.218661 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.218931 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.218933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.218933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.218984 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.219117 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.219118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.219118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.219222 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:03.219310 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:03.219371 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 23:07:03.219427 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260928 23:07:03.221354 22245 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:03.220731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.15.212.65:42919)
0928 23:07:03.220770 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.220772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.220865 (+ 93us) server_negotiation.cc:407] Connection header received
0928 23:07:03.220935 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.220935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.220946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.221011 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.221279 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49282: BlockingRecv error: recv got EOF from 127.0.0.1:49282 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 23:07:03.221520 16209 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc' of type FILE
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.222380 16209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.223237 16209 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:07:03.229800 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.223756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.15.212.65:42919)
0928 23:07:03.223796 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.223797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.223823 (+ 26us) server_negotiation.cc:407] Connection header received
0928 23:07:03.223871 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.223872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.223878 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.223921 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.224242 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.224765 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.225385 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.225567 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.225615 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.227641 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:03.227642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:03.227643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:03.227649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:03.228796 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.229114 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.229115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.229115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.229168 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.229352 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.229353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.229353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.229442 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:03.229539 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:03.229649 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 23:07:03.229715 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 23:07:03.231248 22245 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:03.230825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.15.212.65:42919)
0928 23:07:03.230868 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.230869 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.230907 (+ 38us) server_negotiation.cc:407] Connection header received
0928 23:07:03.230950 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.230951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.230957 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.231000 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.231200 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49290: BlockingRecv error: recv got EOF from 127.0.0.1:49290 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 23:07:03.231338 16209 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790636798761935-16209-0/krb5kdc/krb5cc' of type FILE
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.232189 16209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:03.233050 16209 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[21734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:07:03.238937 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:03.233481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.15.212.65:42919)
0928 23:07:03.233530 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:03.233532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:03.233596 (+ 64us) server_negotiation.cc:407] Connection header received
0928 23:07:03.233631 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:03.233632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:03.233639 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:03.233681 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:03.233963 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.234393 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.234998 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:03.235175 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:03.235225 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:03.236997 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:03.236998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:03.236999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:03.237004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:03.238094 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.238367 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.238367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.238368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.238411 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:03.238548 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:03.238549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:03.238549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:03.238619 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:03.238677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:03.238771 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 23:07:03.238818 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:07:03.239481 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21841
I20260928 23:07:03.244089 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21975
I20260928 23:07:03.248816 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22109
I20260928 23:07:03.253034 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 21747
2026-09-28T23:07:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1811 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[22258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[22258](info): set up 2 sockets
Sep 28 23:07:03 dist-test-slave-x2t8 krb5kdc[22258](info): commencing operation
krb5kdc: starting...
W20260928 23:07:04.377763 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260928 23:07:04.715714 16209 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.338s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260928 23:07:04.949967 16209 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.234s user 0.000s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260928 23:07:05.183437 16209 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.233s user 0.000s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/test-user.keytab.
W20260928 23:07:05.412073 16209 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.229s user 0.000s sys 0.002s
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab.
Entry for principal oryx/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab.
I20260928 23:07:05.455539 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:41451
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab
--principal=oryx/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:41451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:05.537606 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:05.537801 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:05.537822 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:05.538950 22271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:05.538990 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:05.539004 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:05.539040 22271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:05.539068 22271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:05.540550 22271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:41451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab
--principal=oryx/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:41451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:05.540793 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:05.541020 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:05.545744 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:05.545753 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:05.545867 22271 server_base.cc:1034] running on GCE node
W20260928 23:07:05.545760 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:05.546238 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:05.546520 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:05.547662 22271 hybrid_clock.cc:648] HybridClock initialized: now 1790636825547641 us; error 37 us; skew 500 ppm
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:05.554301 22271 init.cc:377] Logged in from keytab as oryx/127.15.212.126@KRBTEST.COM (short username oryx)
I20260928 23:07:05.554755 22271 webserver.cc:460] Webserver started at http://127.15.212.126:34057/ using document root <none> and password file <none>
I20260928 23:07:05.554970 22271 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:05.555013 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:05.555146 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:05.556155 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "65ebb053fe164ab4ae2b365316d730ff"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "6ed792dace97633903d1c95af01dd262"
server_key_iv: "5a2ae917a4e1c98852b5bc2e363111f0"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.556545 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "65ebb053fe164ab4ae2b365316d730ff"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "6ed792dace97633903d1c95af01dd262"
server_key_iv: "5a2ae917a4e1c98852b5bc2e363111f0"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.557847 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:05.558573 22286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.558758 22271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:05.558847 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "65ebb053fe164ab4ae2b365316d730ff"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "6ed792dace97633903d1c95af01dd262"
server_key_iv: "5a2ae917a4e1c98852b5bc2e363111f0"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.558898 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:05.573377 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:05.575546 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:05.575659 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:05.579300 22271 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:41451
I20260928 23:07:05.579370 22338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:41451 every 8 connection(s)
I20260928 23:07:05.579766 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:05.580766 22339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:05.582923 22339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Bootstrap starting.
I20260928 23:07:05.583491 22339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:05.583782 22339 log.cc:824] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:05.584522 22339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: No bootstrap required, opened a new log
I20260928 23:07:05.586058 22339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } }
I20260928 23:07:05.586194 22339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:05.586220 22339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65ebb053fe164ab4ae2b365316d730ff, State: Initialized, Role: FOLLOWER
I20260928 23:07:05.586385 22339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } }
I20260928 23:07:05.586447 22339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:05.586468 22339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:05.586520 22339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:05.587391 22339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } }
I20260928 23:07:05.587476 22339 leader_election.cc:302] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65ebb053fe164ab4ae2b365316d730ff; no voters:
I20260928 23:07:05.587671 22339 leader_election.cc:288] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:05.587728 22342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:05.587886 22342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 LEADER]: Becoming Leader. State: Replica: 65ebb053fe164ab4ae2b365316d730ff, State: Running, Role: LEADER
I20260928 23:07:05.587957 22339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:05.588018 22342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } }
I20260928 23:07:05.588382 22343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } }
I20260928 23:07:05.588500 22343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:05.588387 22344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ebb053fe164ab4ae2b365316d730ff. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } }
I20260928 23:07:05.588608 22344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
W20260928 23:07:05.589753 22357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:05.589798 22357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:05.589861 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22271
I20260928 23:07:05.589862 22358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:05.589972 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:05.590039 22358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:05.590170 16209 external_mini_cluster.cc:1428] Setting key 44fdb8f0e4bd491329fbe370da37f848
I20260928 23:07:05.591866 22358 catalog_manager.cc:1347] Generated new cluster ID: 469391a168a9412d9e54d64e34b39cdb
I20260928 23:07:05.591923 22358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:05.603492 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:05.590623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.15.212.126:41451)
0928 23:07:05.592745 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:05.592754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:05.592788 (+ 34us) server_negotiation.cc:407] Connection header received
0928 23:07:05.592896 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:05.592902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:05.592973 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:05.593090 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:05.593870 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.595260 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.596415 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.597024 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.597114 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:05.599863 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:05.599868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:05.599868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:05.599881 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:05.601867 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.602242 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.602246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.602247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.602358 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.602555 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.602556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.602556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.602877 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:05.603027 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:05.603099 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 23:07:05.603237 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2077,"thread_start_us":74,"threads_started":1}
I20260928 23:07:05.608389 22358 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:05.608884 22358 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:05.620740 22358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Generated new TSK 0
I20260928 23:07:05.620961 22358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
I20260928 23:07:05.655426 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:05.736316 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:05.736532 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:05.736555 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:05.737704 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:05.737746 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:05.737772 22365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:05.739182 22365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:05.739418 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:05.739650 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:05.744603 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:05.744613 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:05.744639 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:05.744642 22365 server_base.cc:1034] running on GCE node
I20260928 23:07:05.745018 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:05.745287 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:05.746436 22365 hybrid_clock.cc:648] HybridClock initialized: now 1790636825746432 us; error 38 us; skew 500 ppm
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:05.748970 22365 init.cc:377] Logged in from keytab as oryx/127.15.212.65@KRBTEST.COM (short username oryx)
I20260928 23:07:05.749336 22365 webserver.cc:460] Webserver started at http://127.15.212.65:41425/ using document root <none> and password file <none>
I20260928 23:07:05.749508 22365 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:05.749547 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:05.749632 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:05.750634 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "dd33f746945d48b3b9737ae7a52f4716"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "70f5dde20319f304bb41df1950690275"
server_key_iv: "26dd811263b247249fa9384414840820"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.750977 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "dd33f746945d48b3b9737ae7a52f4716"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "70f5dde20319f304bb41df1950690275"
server_key_iv: "26dd811263b247249fa9384414840820"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.752249 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:07:05.753015 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.753207 22365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.753304 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "dd33f746945d48b3b9737ae7a52f4716"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "70f5dde20319f304bb41df1950690275"
server_key_iv: "26dd811263b247249fa9384414840820"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.753381 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:05.766752 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:05.768971 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:05.769134 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:05.769354 22365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:05.769644 22365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:05.769695 22365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.769735 22365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:05.769763 22365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.775534 22365 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:35637
I20260928 23:07:05.775580 22493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:35637 every 8 connection(s)
I20260928 23:07:05.775879 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:07:05.780170 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22365
I20260928 23:07:05.780280 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:05.780522 16209 external_mini_cluster.cc:1428] Setting key 5adff7c82933d92e916bf5337a43285f
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:05.785769 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:05.776595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:49751)
0928 23:07:05.776785 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:05.776820 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:05.777319 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:05.777581 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:05.777585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:05.778153 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:05.778339 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:05.778344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.779096 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.779097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:05.779905 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:05.779908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.780313 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.780314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:05.780474 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:05.781029 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:05.781039 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:05.782594 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:05.784498 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:05.784503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:05.784505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:05.784731 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:05.784931 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:05.784931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:05.784931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:05.785024 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:05.785302 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:05.785306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:05.785413 (+ 107us) client_negotiation.cc:769] Sending connection context
0928 23:07:05.785492 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:07:05.785606 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260928 23:07:05.785795 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:05.776621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:49751 (local address 127.15.212.126:41451)
0928 23:07:05.776688 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:05.776690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:05.776903 (+ 213us) server_negotiation.cc:407] Connection header received
0928 23:07:05.777425 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:05.777427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:05.777442 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:05.777512 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:05.778434 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.779005 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.780010 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.780220 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.780305 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:05.782707 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:05.782708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:05.782708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:05.782714 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:05.784393 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.784793 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.784794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.784795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.784861 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.785091 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.785092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.785092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.785236 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:05.785311 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:05.785543 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 23:07:05.785640 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 23:07:05.786298 22494 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:05.786407 22494 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:05.786617 22494 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:05.787175 22303 ts_manager.cc:194] Registered new tserver with Master: dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637)
I20260928 23:07:05.787767 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:49751
WARNING: no policy specified for oryx/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
I20260928 23:07:05.805568 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:05.884876 22499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:05.885072 22499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:05.885094 22499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:05.886152 22499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:05.886183 22499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:05.886209 22499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:05.887624 22499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:05.887799 22499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:05.887960 22499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:05.893085 22505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:05.893121 22504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:05.893117 22507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:05.893106 22499 server_base.cc:1034] running on GCE node
I20260928 23:07:05.893440 22499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:05.893675 22499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:05.894841 22499 hybrid_clock.cc:648] HybridClock initialized: now 1790636825894828 us; error 29 us; skew 500 ppm
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:05.897202 22499 init.cc:377] Logged in from keytab as oryx/127.15.212.66@KRBTEST.COM (short username oryx)
I20260928 23:07:05.897663 22499 webserver.cc:460] Webserver started at http://127.15.212.66:36441/ using document root <none> and password file <none>
I20260928 23:07:05.897886 22499 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:05.897951 22499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:05.898092 22499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:05.899098 22499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "c0e2059b4fee46f296d35384db3e6a75"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "e90b7d5eabe6992522612fc99c34b9ba"
server_key_iv: "bbdce1e6140242fc5cb52e19fcf7d856"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.899457 22499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "c0e2059b4fee46f296d35384db3e6a75"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "e90b7d5eabe6992522612fc99c34b9ba"
server_key_iv: "bbdce1e6140242fc5cb52e19fcf7d856"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.900795 22499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:07:05.901465 22514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.901654 22499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:05.901757 22499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "c0e2059b4fee46f296d35384db3e6a75"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "e90b7d5eabe6992522612fc99c34b9ba"
server_key_iv: "bbdce1e6140242fc5cb52e19fcf7d856"
server_key_version: "encryptionkey@0"
I20260928 23:07:05.901834 22499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:05.918694 22499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:05.920837 22499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:05.920995 22499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:05.921231 22499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:05.921530 22499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:05.921562 22499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.921607 22499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:05.921636 22499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:05.927177 22499 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:45219
I20260928 23:07:05.927237 22627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:45219 every 8 connection(s)
I20260928 23:07:05.927526 22499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:05.929965 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22499
I20260928 23:07:05.930092 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:05.930279 16209 external_mini_cluster.cc:1428] Setting key c321577481ccb30f084b05e3b61e9390
Sep 28 23:07:05 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:05.937363 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:05.928344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:54579 (local address 127.15.212.126:41451)
0928 23:07:05.928406 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:05.928408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:05.928616 (+ 208us) server_negotiation.cc:407] Connection header received
0928 23:07:05.929077 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:05.929079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:05.929087 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:05.929155 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:05.930177 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.930766 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.932258 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.932479 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.932563 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:05.934607 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:05.934608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:05.934609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:05.934616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:05.935975 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.936384 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.936385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.936385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.936488 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:05.936740 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:05.936741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:05.936741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:05.936858 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:05.936935 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:05.937191 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 23:07:05.937275 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:07:05.937572 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:05.928297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:54579)
0928 23:07:05.928479 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:05.928528 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:05.928998 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:05.929218 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:05.929221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:05.929790 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:05.930081 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:05.930089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.930867 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.930868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:05.932180 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:05.932183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:05.932569 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:05.932570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:05.932699 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:05.933206 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:05.933213 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:05.934513 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:05.936078 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:05.936084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:05.936086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:05.936310 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:05.936563 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:05.936564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:05.936564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:05.936674 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:05.936925 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:05.936931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:05.937065 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 23:07:05.937170 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 23:07:05.937317 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260928 23:07:05.938041 22628 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:05.938131 22628 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:07:05.938319 22628 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:05.938761 22303 ts_manager.cc:194] Registered new tserver with Master: c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:05.939110 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:54579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
I20260928 23:07:05.956312 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:06.035310 22633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.035498 22633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.035528 22633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.036592 22633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.036628 22633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.036651 22633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:06.038182 22633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:06.038362 22633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:06.038535 22633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.043561 22638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.043529 22639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.043533 22641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.043563 22633 server_base.cc:1034] running on GCE node
I20260928 23:07:06.043958 22633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.044180 22633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.045343 22633 hybrid_clock.cc:648] HybridClock initialized: now 1790636826045331 us; error 23 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.047704 22633 init.cc:377] Logged in from keytab as oryx/127.15.212.67@KRBTEST.COM (short username oryx)
I20260928 23:07:06.048143 22633 webserver.cc:460] Webserver started at http://127.15.212.67:45147/ using document root <none> and password file <none>
I20260928 23:07:06.048378 22633 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.048465 22633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.048617 22633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:06.049614 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "38049d9a357149309c1e5891272a6c2b"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a25bf188fe346851be49c1c342b3a5f9"
server_key_iv: "8ec0614788179aa52a61a8eef0796415"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.049978 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "38049d9a357149309c1e5891272a6c2b"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a25bf188fe346851be49c1c342b3a5f9"
server_key_iv: "8ec0614788179aa52a61a8eef0796415"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.051254 22633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:06.051928 22648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.052088 22633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:06.052181 22633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "38049d9a357149309c1e5891272a6c2b"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a25bf188fe346851be49c1c342b3a5f9"
server_key_iv: "8ec0614788179aa52a61a8eef0796415"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.052258 22633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.065533 22633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.067811 22633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:06.067970 22633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.068203 22633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:06.068554 22633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:06.068606 22633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.068645 22633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:06.068673 22633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.074546 22633 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:37281
I20260928 23:07:06.074584 22761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:37281 every 8 connection(s)
I20260928 23:07:06.074896 22633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:06.080662 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22633
I20260928 23:07:06.080766 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:06.080940 16209 external_mini_cluster.cc:1428] Setting key 8871dba2d41e427b9463ebe968998fd3
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:06.083940 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.075585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:47607 (local address 127.15.212.126:41451)
0928 23:07:06.075670 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.075672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.075845 (+ 173us) server_negotiation.cc:407] Connection header received
0928 23:07:06.076370 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.076372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.076385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.076449 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.077360 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.077871 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.078851 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.079061 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.079145 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.081491 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.081492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.081492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.081515 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.082843 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.083201 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.083202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.083202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.083251 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.083453 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.083454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.083454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.083550 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.083598 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.083818 (+ 220us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.083878 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:07:06.084193 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.075567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:47607)
0928 23:07:06.075731 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.075765 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.076261 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.076534 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.076538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.077074 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.077264 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.077268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.077964 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.077965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.078761 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.078763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.079158 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.079159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.079293 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.079822 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.079829 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.081397 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.082909 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.082914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.082915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.083126 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.083308 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.083308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.083309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.083400 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.083601 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.083604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.083715 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.083793 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.083901 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260928 23:07:06.084779 22762 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:06.084878 22762 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.085041 22762 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:06.085384 22303 ts_manager.cc:194] Registered new tserver with Master: 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:06.085719 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:47607
I20260928 23:07:06.092960 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.212.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.125.keytab.
Entry for principal oryx/127.15.212.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.125.keytab.
I20260928 23:07:06.116521 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:40431
--webserver_interface=127.15.212.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.125.keytab
--principal=oryx/127.15.212.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:41451,127.15.212.125:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:06.197213 22767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.197438 22767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.197494 22767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.198604 22767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:06.198673 22767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.198695 22767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.198716 22767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:06.198747 22767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:06.200162 22767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:41451,127.15.212.125:40431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.125.keytab
--principal=oryx/127.15.212.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.212.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
W20260928 23:07:06.200309 22767 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.212.126:41451,127.15.212.125:40431'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 23:07:06.200484 22767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:06.200702 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.205346 22775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.205355 22772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.205354 22773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.205519 22767 server_base.cc:1034] running on GCE node
I20260928 23:07:06.205804 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.206084 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.207254 22767 hybrid_clock.cc:648] HybridClock initialized: now 1790636826207232 us; error 41 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.209658 22767 init.cc:377] Logged in from keytab as oryx/127.15.212.125@KRBTEST.COM (short username oryx)
I20260928 23:07:06.210112 22767 webserver.cc:460] Webserver started at http://127.15.212.125:38709/ using document root <none> and password file <none>
I20260928 23:07:06.210348 22767 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.210391 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.210534 22767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:06.211532 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/instance:
uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "2c78847f5867aa832e04ef368361a109"
server_key_iv: "5f829d5adbbf1ac2fe908f5dc1ab8ac0"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.211882 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal/instance:
uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "2c78847f5867aa832e04ef368361a109"
server_key_iv: "5f829d5adbbf1ac2fe908f5dc1ab8ac0"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.213223 22767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:06.213927 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.214087 22767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:06.214196 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "2c78847f5867aa832e04ef368361a109"
server_key_iv: "5f829d5adbbf1ac2fe908f5dc1ab8ac0"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.214270 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.236771 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.238758 22767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:06.238931 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.242801 22767 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.125:40431
I20260928 23:07:06.242835 22834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.125:40431 every 8 connection(s)
I20260928 23:07:06.243208 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
I20260928 23:07:06.244190 22835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:06.245699 22835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:07:06.250715 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22767
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.125@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:06.254845 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.246360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47172 (local address 127.15.212.126:41451)
0928 23:07:06.246443 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.246446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.246600 (+ 154us) server_negotiation.cc:407] Connection header received
0928 23:07:06.247020 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.247021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.247032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.247079 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.247635 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.248106 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.249135 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.249317 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.249369 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.251540 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.251542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.251542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.251550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.253211 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.253676 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.253677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.253677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.253757 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.254055 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.254056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.254056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.254255 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.254322 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.254677 (+ 355us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.254755 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 23:07:06.255350 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.246303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47172)
0928 23:07:06.246484 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.246520 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.246941 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.247144 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.247147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.247386 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.247570 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.247574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.248180 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.248181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.249064 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.249066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.249387 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.249388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.249489 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.249985 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.249992 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.251445 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.253289 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.253294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.253296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.253598 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.253843 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.253844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.253844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.253995 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.254352 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.254358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.254509 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.254975 (+ 466us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.255135 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260928 23:07:06.255977 22835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.125@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:06.260893 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.251196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47800 (local address 127.15.212.125:40431)
0928 23:07:06.251353 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.251357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.251383 (+ 26us) server_negotiation.cc:407] Connection header received
0928 23:07:06.251438 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.251441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.251455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.251521 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.252103 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.253276 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.254320 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.255036 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.255219 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.258090 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.258097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.258097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.258102 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.259663 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.260055 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.260057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.260058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.260120 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.260330 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.260331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.260331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.260521 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.260670 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.260717 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.260790 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260928 23:07:06.263581 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.256226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47802)
0928 23:07:06.256365 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.256374 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.256409 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.256928 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.256929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.257157 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.257207 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.257209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.257833 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.257834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.258539 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.258542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.259063 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.259064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.259116 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.259473 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.259477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.260977 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.262631 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.262631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.262632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.262787 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.262991 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.262991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.262992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.263043 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.263282 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.263283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.263317 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.263384 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.263472 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260928 23:07:06.263665 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.256296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.15.212.125:40431)
0928 23:07:06.256792 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.256794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.256802 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.256820 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.256821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.256830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.256897 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.257282 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.257756 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.258619 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.258988 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.259057 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.261059 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.261061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.261061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.261067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.262532 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.262868 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.262869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.262869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.262920 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.263111 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.263111 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.263112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.263218 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.263290 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.263410 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.263521 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":477,"thread_start_us":95,"threads_started":1}
I20260928 23:07:06.264894 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Bootstrap starting.
I20260928 23:07:06.265492 22835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:06.265762 22835 log.cc:824] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:06.266460 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: No bootstrap required, opened a new log
I20260928 23:07:06.267772 22835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } }
I20260928 23:07:06.267884 22835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:06.267906 22835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8663db9cbc6c4d1b9c7070a81a99b9cc, State: Initialized, Role: FOLLOWER
I20260928 23:07:06.268044 22835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } }
I20260928 23:07:06.268337 22835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:06.268306 22843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } } }
I20260928 23:07:06.268445 22843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: FOLLOWER
W20260928 23:07:06.269961 22854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:06.270020 22854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:06.271543 22767 master_runner.cc:428] Detected that this master 8663db9cbc6c4d1b9c7070a81a99b9cc is joining an existing cluster
I20260928 23:07:06.271608 22767 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.212.125:40431
I20260928 23:07:06.278764 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.272840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.15.212.126:41451)
0928 23:07:06.272903 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.272905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.272913 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.272935 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.272936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.272945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.272997 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.273397 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.273881 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.274625 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.274824 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.274893 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.275825 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.275827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.275827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.275834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.277464 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.277906 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.277907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.277907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.277971 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278185 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.278186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.278186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.278314 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.278381 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.278575 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.278661 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:07:06.279203 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.272489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47186)
0928 23:07:06.272641 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.272650 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.272695 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.273034 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.273035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.273254 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.273311 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.273312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.273966 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.273967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.274530 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.274531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.274925 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.274926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.274986 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.275436 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.275441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.275751 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.277564 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.277565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.277565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.277829 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.278046 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.278048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.278115 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.278409 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.278410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.278460 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.278843 (+ 383us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.278941 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260928 23:07:06.279333 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.272882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.15.212.125:40431)
0928 23:07:06.272914 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.272916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.272924 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.272942 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.272942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.272951 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.273005 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.273446 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.274169 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.275168 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.275452 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.275491 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.276514 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.276515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.276516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.276522 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.278230 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278549 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.278550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.278551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.278598 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278798 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.278799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.278799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.278915 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.278998 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.279143 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.279213 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:07:06.279580 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.272723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47812)
0928 23:07:06.272830 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.272837 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.272883 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.273043 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.273044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.273296 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.273360 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.273362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.274255 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.274256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.275075 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.275077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.275554 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.275555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.275607 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.276015 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.276020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.276368 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.278313 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.278314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.278469 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.278668 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.278668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.278669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.278727 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.278989 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.278990 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.279029 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.279401 (+ 372us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.279476 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":71,"threads_started":1}
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.126@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:06.288738 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.281568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47818)
0928 23:07:06.281676 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.281718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.281793 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.281924 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.281926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.282156 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.282205 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.282206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.283039 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.283040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.283591 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.283593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.284027 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.284028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.284079 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.284537 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.284542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.285980 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.287607 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.287614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.287615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.287812 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.288001 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.288001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.288002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.288054 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.288291 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.288296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.288335 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.288507 (+ 172us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.288652 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
I20260928 23:07:06.288733 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.281642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.15.212.125:40431)
0928 23:07:06.281679 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.281681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.281782 (+ 101us) server_negotiation.cc:407] Connection header received
0928 23:07:06.281847 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.281847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.281856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.281921 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.282268 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.282963 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.283667 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.283956 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.284024 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.286069 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.286070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.286071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.286078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.287537 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.287881 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.287882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.287882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.287937 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.288110 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.288110 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.288111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.288230 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.288295 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.288535 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.288607 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:07:06.289340 22303 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.212.125:40431: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
member_type: NON_VOTER
last_known_addr {
host: "127.15.212.125"
port: 40431
}
attrs {
promote: true
}
}
dest_uuid: "65ebb053fe164ab4ae2b365316d730ff"
cas_config_opid_index: -1
I20260928 23:07:06.289676 22303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } }
I20260928 23:07:06.290158 22864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } }
I20260928 23:07:06.290472 22809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:06.291201 22809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Refusing update from remote peer 65ebb053fe164ab4ae2b365316d730ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 23:07:06.291231 22864 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.212.125:40431
I20260928 23:07:06.291441 22767 master.cc:537] Master@127.15.212.125:40431 shutting down...
I20260928 23:07:06.291646 22865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:06.292203 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } }
I20260928 23:07:06.292292 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
W20260928 23:07:06.292510 22289 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 23:07:06.293509 22289 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff -> Peer 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125:40431): Couldn't send request to peer 8663db9cbc6c4d1b9c7070a81a99b9cc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 23:07:06.294883 22767 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:06.294997 22767 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:06.295050 22767 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: stopping tablet replica
I20260928 23:07:06.307432 22767 master.cc:552] Master@127.15.212.125:40431 shutdown complete.
I20260928 23:07:06.307499 22767 master_runner.cc:305] Clearing existing system tablet
I20260928 23:07:06.307621 22767 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.307682 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.308319 22767 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:06.308866 22869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.308979 22767 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.309034 22767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.309056 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "2c78847f5867aa832e04ef368361a109"
server_key_iv: "5f829d5adbbf1ac2fe908f5dc1ab8ac0"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.309662 22767 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:06.311167 22767 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 23:07:06.311233 22767 log.cc:1196] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 23:07:06.311441 22767 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Deleting consensus metadata
I20260928 23:07:06.311566 22767 master_runner.cc:315] Copying system tablet from 127.15.212.126:41451
I20260928 23:07:06.312165 22767 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: tablet copy: Beginning tablet copy session from remote peer at address 127.15.212.126:41451
I20260928 23:07:06.318183 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.312474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.15.212.126:41451)
0928 23:07:06.312513 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.312515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.312645 (+ 130us) server_negotiation.cc:407] Connection header received
0928 23:07:06.312719 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.312720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.312731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.312788 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.313102 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.313594 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.314325 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.314532 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.314603 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.315571 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.315573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.315573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.315580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.317017 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.317388 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.317389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.317390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.317448 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.317657 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.317658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.317659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.317773 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.317844 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.318026 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.318095 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:07:06.318204 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.312425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47190)
0928 23:07:06.312558 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.312567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.312661 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.312793 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.312793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.312962 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.313015 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.313016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.313675 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.313676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.314238 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.314240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.314614 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.314614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.314686 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.315159 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.315164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.315460 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.317121 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.317122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.317122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.317302 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.317527 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.317528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.317529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.317587 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.317847 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.317848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.317891 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.317985 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.318091 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":96,"threads_started":1}
I20260928 23:07:06.318463 22323 tablet_copy_service.cc:140] P 65ebb053fe164ab4ae2b365316d730ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8663db9cbc6c4d1b9c7070a81a99b9cc ({username='oryx', principal='oryx/127.15.212.125@KRBTEST.COM'} at 127.0.0.1:47190)
I20260928 23:07:06.318567 22323 tablet_copy_service.cc:161] P 65ebb053fe164ab4ae2b365316d730ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8663db9cbc6c4d1b9c7070a81a99b9cc at {username='oryx', principal='oryx/127.15.212.125@KRBTEST.COM'} at 127.0.0.1:47190: session id = 8663db9cbc6c4d1b9c7070a81a99b9cc-00000000000000000000000000000000
I20260928 23:07:06.319161 22323 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 23:07:06.319763 22767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 23:07:06.320714 22767 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: tablet copy: Starting download of 0 data blocks...
I20260928 23:07:06.320875 22767 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: tablet copy: Starting download of 1 WAL segments...
I20260928 23:07:06.321710 22767 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 23:07:06.322412 22323 tablet_copy_service.cc:342] P 65ebb053fe164ab4ae2b365316d730ff: Request end of tablet copy session 8663db9cbc6c4d1b9c7070a81a99b9cc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.212.125@KRBTEST.COM'} at 127.0.0.1:47190
I20260928 23:07:06.322472 22323 tablet_copy_service.cc:433] P 65ebb053fe164ab4ae2b365316d730ff: ending tablet copy session 8663db9cbc6c4d1b9c7070a81a99b9cc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8663db9cbc6c4d1b9c7070a81a99b9cc
I20260928 23:07:06.324887 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.326025 22876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.326102 22877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.326104 22879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.326347 22767 server_base.cc:1034] running on GCE node
I20260928 23:07:06.326436 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.326582 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.327721 22767 hybrid_clock.cc:648] HybridClock initialized: now 1790636826327706 us; error 30 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.329097 22767 init.cc:377] Logged in from keytab as oryx/127.15.212.125@KRBTEST.COM (short username oryx)
I20260928 23:07:06.329334 22767 webserver.cc:460] Webserver started at http://127.15.212.125:35121/ using document root <none> and password file <none>
I20260928 23:07:06.329447 22767 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.329483 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.330152 22767 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.330752 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.330938 22767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.331014 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "2c78847f5867aa832e04ef368361a109"
server_key_iv: "5f829d5adbbf1ac2fe908f5dc1ab8ac0"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.331070 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.342471 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.344097 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.347216 22767 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.125:40431
I20260928 23:07:06.347610 22938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.125:40431 every 8 connection(s)
I20260928 23:07:06.347632 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
I20260928 23:07:06.348333 22939 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:07:06.348809 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Bootstrap starting.
I20260928 23:07:06.351063 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:07:06.351291 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: Bootstrap complete.
I20260928 23:07:06.351480 22939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } }
I20260928 23:07:06.351564 22939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8663db9cbc6c4d1b9c7070a81a99b9cc, State: Initialized, Role: LEARNER
I20260928 23:07:06.351613 22939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } }
I20260928 23:07:06.351780 22939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:06.351773 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } }
I20260928 23:07:06.351871 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: LEARNER
I20260928 23:07:06.353304 22953 catalog_manager.cc:1259] Loaded cluster ID: 469391a168a9412d9e54d64e34b39cdb
I20260928 23:07:06.353360 22953 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: loading cluster ID for follower catalog manager: success
I20260928 23:07:06.354308 22953 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: acquiring CA information for follower catalog manager: success
I20260928 23:07:06.354605 22953 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:07:06.368999 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.362732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.15.212.125:40431)
0928 23:07:06.362846 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.362848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.362861 (+ 13us) server_negotiation.cc:407] Connection header received
0928 23:07:06.362882 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.362883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.362891 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.362962 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.363476 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.364119 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.364887 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.365091 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.365167 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.366356 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.366358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.366358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.366364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.367796 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.368155 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.368156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.368156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.368215 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.368418 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.368419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.368419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.368565 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.368673 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.368781 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.368860 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for oryx/127.15.212.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.124.keytab.
Entry for principal oryx/127.15.212.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.124.keytab.
I20260928 23:07:06.399303 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:46229
--webserver_interface=127.15.212.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.124.keytab
--principal=oryx/127.15.212.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:06.493628 22957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.493816 22957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.493839 22957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.494987 22957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:06.495028 22957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.495042 22957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.495055 22957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:06.495067 22957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:06.496651 22957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.124.keytab
--principal=oryx/127.15.212.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:46229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.212.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:06.496883 22957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:06.497117 22957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.501869 22962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.501869 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.501941 22957 server_base.cc:1034] running on GCE node
W20260928 23:07:06.502141 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.502386 22957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.502688 22957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.503856 22957 hybrid_clock.cc:648] HybridClock initialized: now 1790636826503848 us; error 34 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.506660 22957 init.cc:377] Logged in from keytab as oryx/127.15.212.124@KRBTEST.COM (short username oryx)
I20260928 23:07:06.507196 22957 webserver.cc:460] Webserver started at http://127.15.212.124:42475/ using document root <none> and password file <none>
I20260928 23:07:06.507454 22957 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.507521 22957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.507661 22957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:06.509168 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/instance:
uuid: "3773c0660e474c498d93334f2113cc3f"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a33cb16646778538cd3639c2fb7c0326"
server_key_iv: "d2f8eb988513ec8fc98f862c95a54865"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.509578 22957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal/instance:
uuid: "3773c0660e474c498d93334f2113cc3f"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a33cb16646778538cd3639c2fb7c0326"
server_key_iv: "d2f8eb988513ec8fc98f862c95a54865"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.511258 22957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:06.512115 22972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.512333 22957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:06.512425 22957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "3773c0660e474c498d93334f2113cc3f"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a33cb16646778538cd3639c2fb7c0326"
server_key_iv: "d2f8eb988513ec8fc98f862c95a54865"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.512521 22957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.530195 22957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.532670 22957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:06.532816 22957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.537743 22957 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.124:46229
I20260928 23:07:06.537766 23024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.124:46229 every 8 connection(s)
I20260928 23:07:06.538180 22957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
I20260928 23:07:06.539193 23025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:06.540791 23025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:07:06.544636 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 22957
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.124@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
I20260928 23:07:06.550374 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.541658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47192 (local address 127.15.212.126:41451)
0928 23:07:06.541707 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.541709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.541852 (+ 143us) server_negotiation.cc:407] Connection header received
0928 23:07:06.542355 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.542357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.542370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.542461 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.543047 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.543535 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.544552 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.544770 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.544838 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.547303 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.547304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.547304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.547310 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.549055 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.549433 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.549434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.549434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.549503 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.549738 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.549739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.549740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.549853 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.549932 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.550208 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.550286 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:07:06.550537 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.541502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47192)
0928 23:07:06.541717 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.541759 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.542251 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.542533 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.542537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.542773 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.542949 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.542953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.543627 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.543628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.544420 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.544422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.544875 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.544876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.545022 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.545604 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.545615 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.547191 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.549144 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.549148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.549150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.549365 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.549568 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.549568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.549568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.549657 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.549918 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.549924 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.550060 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.550174 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.550312 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260928 23:07:06.551056 23025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:07:06.553984 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.545276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.15.212.124:46229)
0928 23:07:06.545411 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.545416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.545436 (+ 20us) server_negotiation.cc:407] Connection header received
0928 23:07:06.545457 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.545459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.545479 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.545557 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.546285 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.547008 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.547741 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.548058 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.548141 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.550820 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.550824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.550824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.550829 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.552729 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.553077 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.553079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.553080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.553138 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.553368 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.553369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.553370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.553571 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.553710 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.553806 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.553876 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.124@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:06.558598 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.551295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47864)
0928 23:07:06.551339 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.551348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.551378 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.551516 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.551517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.551640 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.551679 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.551680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.552416 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.552417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.553283 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.553284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.554460 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.554461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.554537 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.554816 (+ 279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.554820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.555914 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.557611 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.557612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.557612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.557955 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.558136 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.558137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.558137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.558190 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.558394 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.558395 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.558427 (+ 32us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.558474 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.558536 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:07:06.558619 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.551359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.15.212.125:40431)
0928 23:07:06.551404 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.551406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.551413 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:06.551437 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.551437 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.551449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.551501 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.551779 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.552333 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.553394 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.553719 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.553783 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.556031 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.556032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.556033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.556039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.557536 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.558027 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.558028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.558028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.558077 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.558245 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.558245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.558246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.558340 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.558389 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.558497 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.558559 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:07:06.559121 23025 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.124@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
I20260928 23:07:06.564900 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.559358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39264)
0928 23:07:06.559405 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.559416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.559454 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.559564 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.559565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.559664 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.559689 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.559690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.560241 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.560242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.560772 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.560774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.561118 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.561119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.561154 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.561484 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.561488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.562668 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.564151 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.564151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.564151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.564291 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.564416 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.564416 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.564416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.564506 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.564681 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.564681 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.564715 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.564764 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.564829 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 23:07:06.565110 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.559419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39264 (local address 127.15.212.124:46229)
0928 23:07:06.559467 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.559468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.559476 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.559504 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.559505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.559519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.559562 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.559731 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.560156 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.560839 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.561058 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.561117 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.562742 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.562743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.562743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.562749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.564079 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.564343 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.564343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.564344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.564381 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.564544 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.564545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.564545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.564627 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.564675 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.564779 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.564870 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":31}
I20260928 23:07:06.566329 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Bootstrap starting.
I20260928 23:07:06.567087 23025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:06.567358 23025 log.cc:824] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:06.568009 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: No bootstrap required, opened a new log
I20260928 23:07:06.569409 23025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } }
I20260928 23:07:06.569543 23025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:06.569589 23025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3773c0660e474c498d93334f2113cc3f, State: Initialized, Role: FOLLOWER
I20260928 23:07:06.569722 23025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } }
I20260928 23:07:06.569998 23025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:06.569965 23030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } } }
I20260928 23:07:06.570083 23030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: FOLLOWER
W20260928 23:07:06.571600 23041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:06.571655 23041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:06.573814 22957 master_runner.cc:428] Detected that this master 3773c0660e474c498d93334f2113cc3f is joining an existing cluster
I20260928 23:07:06.573889 22957 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.212.124:46229
I20260928 23:07:06.581996 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.575273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39266)
0928 23:07:06.575841 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.575850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.575904 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.576029 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.576029 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.576202 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.576246 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.576247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.577134 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.577897 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.577899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.578357 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.578358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.578429 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.578824 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.578829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.579136 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.580958 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.580959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.580960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.581169 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.581382 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.581382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.581383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.581457 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.581727 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.581728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.581771 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.581844 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.581927 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":72,"threads_started":1}
I20260928 23:07:06.582105 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.575306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39266 (local address 127.15.212.124:46229)
0928 23:07:06.575342 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.575343 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.575887 (+ 544us) server_negotiation.cc:407] Connection header received
0928 23:07:06.575959 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.575959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.575966 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.575989 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.576312 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.577046 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577976 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.578249 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.578292 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.579249 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.579250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.579251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.579257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.580873 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.581249 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.581250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.581251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.581309 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.581524 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.581525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.581525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.581652 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.581728 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.581882 (+ 154us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.581989 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":20}
I20260928 23:07:06.604050 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.574976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.15.212.125:40431)
0928 23:07:06.575021 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.575023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.575031 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.575051 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.575051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.575061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.575132 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.575613 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.576374 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577394 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.577711 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577769 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.600956 (+ 23187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.600959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.600959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.600969 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.602798 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603188 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.603189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.603190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.603249 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603470 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.603471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.603472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.603617 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.603698 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.603865 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.603953 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:07:06.604079 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.574818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47880)
0928 23:07:06.574934 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.574944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.574986 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.575343 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.575344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.575474 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.575533 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.575534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.576473 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.576474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.577314 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.577315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.577829 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.577883 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.578300 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.578304 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.600832 (+ 22528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.602892 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.602893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.602893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.603108 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.603316 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.603317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.603375 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.603686 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.603687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.603751 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.603824 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.603943 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260928 23:07:06.604262 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.574765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47196)
0928 23:07:06.574910 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.574921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.574997 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.575156 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.575157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.575672 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.575728 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.575729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.576656 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.576657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.577546 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.577549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.578146 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.578147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.578210 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.601051 (+ 22841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.601055 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.601436 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.603143 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.603145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.603356 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.603584 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.603585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.603637 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.603950 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.603950 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.603985 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.604043 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.604146 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260928 23:07:06.604684 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.574806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.15.212.126:41451)
0928 23:07:06.574848 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.574851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.574975 (+ 124us) server_negotiation.cc:407] Connection header received
0928 23:07:06.575081 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.575082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.575091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.575138 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.575805 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.576561 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.577745 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.578034 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.578096 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.601522 (+ 23426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.601523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.601524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.601532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.603051 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603450 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.603451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.603452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.603504 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.603729 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.603730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.603730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.603865 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.603924 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.604480 (+ 556us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.604556 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:07:06.605201 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22365
I20260928 23:07:06.610610 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:35637
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:41451
--webserver_port=41425
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636825, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.126@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
I20260928 23:07:06.616421 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.606706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39274)
0928 23:07:06.606753 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.606763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.606799 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.609427 (+ 2628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.609428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.609601 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.609648 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.609650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.610540 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.610541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.611238 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.611240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.611638 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.611639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.611706 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.612195 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.612202 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.613767 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.615362 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.615363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.615363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.615535 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.615763 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.615763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.615764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.615824 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.616075 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.616076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.616120 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.616213 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.616323 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:07:06.616418 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.609219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.15.212.124:46229)
0928 23:07:06.609256 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.609258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.609266 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:06.609303 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.609304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.609313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.609370 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.609751 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.610455 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.611323 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.611549 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.611644 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.613855 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.613857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.613857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.613864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.615279 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.615619 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.615620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.615621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.615680 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.615902 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.615903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.615903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.616008 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.616083 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.616256 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.616324 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:07:06.617034 22303 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.212.124:46229: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3773c0660e474c498d93334f2113cc3f"
member_type: NON_VOTER
last_known_addr {
host: "127.15.212.124"
port: 46229
}
attrs {
promote: true
}
}
dest_uuid: "65ebb053fe164ab4ae2b365316d730ff"
cas_config_opid_index: 5
I20260928 23:07:06.617287 22303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } }
I20260928 23:07:06.617648 22865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3773c0660e474c498d93334f2113cc3f (127.15.212.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.618448 22865 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.212.124:46229
I20260928 23:07:06.618526 22865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.618597 22865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:06.618994 22957 master.cc:537] Master@127.15.212.124:46229 shutting down...
W20260928 23:07:06.619398 22287 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff -> Peer 3773c0660e474c498d93334f2113cc3f (127.15.212.124:46229): Couldn't send request to peer 3773c0660e474c498d93334f2113cc3f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 23:07:06.622267 22957 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 0 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:06.622355 22957 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 0 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:06.622399 22957 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: stopping tablet replica
I20260928 23:07:06.624176 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.618861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.15.212.125:40431)
0928 23:07:06.618902 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.618905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.619512 (+ 607us) server_negotiation.cc:407] Connection header received
0928 23:07:06.619606 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.619607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.619630 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.619671 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:06.619863 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.620614 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.623077 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.623688 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.623738 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.623820 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.624055 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.624099 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:07:06.624395 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.618801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:47894)
0928 23:07:06.619462 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.619472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.619537 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.619745 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.619746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.619747 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:06.619789 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.619791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.620709 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.620710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.622967 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.622971 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.623878 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.623878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.623949 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.623972 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.624281 (+ 309us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.624328 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645}
I20260928 23:07:06.624647 22913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 LEARNER]: Refusing update from remote peer 65ebb053fe164ab4ae2b365316d730ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 23:07:06.624943 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 23:07:06.625523 22913 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3773c0660e474c498d93334f2113cc3f (127.15.212.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.626571 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ebb053fe164ab4ae2b365316d730ff. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.626662 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: LEARNER
I20260928 23:07:06.626708 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: NON_VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: true } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.626761 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: LEARNER
I20260928 23:07:06.626983 22863 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: attempting to promote NON_VOTER 8663db9cbc6c4d1b9c7070a81a99b9cc to VOTER
I20260928 23:07:06.627161 22863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } }
I20260928 23:07:06.627640 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.627526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39282)
0928 23:07:06.627572 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.627614 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:46229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 23:07:06.627709 22913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 LEARNER]: Refusing update from remote peer 65ebb053fe164ab4ae2b365316d730ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 23:07:06.627923 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 23:07:06.628058 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.627975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39292)
0928 23:07:06.628008 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.628031 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:46229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260928 23:07:06.628189 22287 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff -> Peer 3773c0660e474c498d93334f2113cc3f (127.15.212.124:46229): Couldn't send request to peer 3773c0660e474c498d93334f2113cc3f. Status: Network error: Client connection negotiation failed: client connection to 127.15.212.124:46229: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 23:07:06.628479 22865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.628657 22913 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.629377 22865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.629463 22865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:06.629493 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.629556 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:06.629686 23052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.629766 23052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:06.635592 22957 master.cc:552] Master@127.15.212.124:46229 shutdown complete.
I20260928 23:07:06.635654 22957 master_runner.cc:305] Clearing existing system tablet
I20260928 23:07:06.635790 22957 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.635846 22957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.636466 22957 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:06.636961 23054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.637111 22957 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.637223 22957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:06.637257 22957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "3773c0660e474c498d93334f2113cc3f"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a33cb16646778538cd3639c2fb7c0326"
server_key_iv: "d2f8eb988513ec8fc98f862c95a54865"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.637866 22957 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:06.639183 22957 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 23:07:06.639250 22957 log.cc:1196] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 23:07:06.639436 22957 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Deleting consensus metadata
I20260928 23:07:06.639559 22957 master_runner.cc:315] Copying system tablet from 127.15.212.126:41451
I20260928 23:07:06.640134 22957 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.212.126:41451
I20260928 23:07:06.646191 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.640612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.15.212.126:41451)
0928 23:07:06.640663 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.640666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.640778 (+ 112us) server_negotiation.cc:407] Connection header received
0928 23:07:06.640852 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.640853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.640863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.640918 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.641222 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.641704 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.642401 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.642597 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.642682 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.643623 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.643625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.643625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.643632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.645051 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.645433 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.645434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.645435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.645490 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.645691 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.645692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.645692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.645803 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.645877 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.646031 (+ 154us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.646104 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:07:06.646219 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.640579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:47202)
0928 23:07:06.640706 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.640715 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.640791 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.640923 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.640924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.641094 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.641134 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.641136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.641783 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.641784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.642306 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.642307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.642680 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.642681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.642747 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.643213 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.643218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.643512 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.645145 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.645146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.645147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.645357 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.645560 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.645561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.645562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.645619 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.645866 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.645867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.645919 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.646005 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.646104 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260928 23:07:06.646651 22323 tablet_copy_service.cc:140] P 65ebb053fe164ab4ae2b365316d730ff: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3773c0660e474c498d93334f2113cc3f ({username='oryx', principal='oryx/127.15.212.124@KRBTEST.COM'} at 127.0.0.1:47202)
I20260928 23:07:06.646705 22323 tablet_copy_service.cc:161] P 65ebb053fe164ab4ae2b365316d730ff: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3773c0660e474c498d93334f2113cc3f at {username='oryx', principal='oryx/127.15.212.124@KRBTEST.COM'} at 127.0.0.1:47202: session id = 3773c0660e474c498d93334f2113cc3f-00000000000000000000000000000000
I20260928 23:07:06.647121 22323 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 23:07:06.647552 22957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 23:07:06.648548 22957 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: tablet copy: Starting download of 0 data blocks...
I20260928 23:07:06.648702 22957 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: tablet copy: Starting download of 1 WAL segments...
I20260928 23:07:06.649326 22957 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 23:07:06.650023 22323 tablet_copy_service.cc:342] P 65ebb053fe164ab4ae2b365316d730ff: Request end of tablet copy session 3773c0660e474c498d93334f2113cc3f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.212.124@KRBTEST.COM'} at 127.0.0.1:47202
I20260928 23:07:06.650072 22323 tablet_copy_service.cc:433] P 65ebb053fe164ab4ae2b365316d730ff: ending tablet copy session 3773c0660e474c498d93334f2113cc3f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3773c0660e474c498d93334f2113cc3f
I20260928 23:07:06.652235 22957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.653473 23062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.653482 23061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.653887 23064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.657375 22957 server_base.cc:1034] running on GCE node
I20260928 23:07:06.657486 22957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.657655 22957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.658779 22957 hybrid_clock.cc:648] HybridClock initialized: now 1790636826658778 us; error 19 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.660058 22957 init.cc:377] Logged in from keytab as oryx/127.15.212.124@KRBTEST.COM (short username oryx)
I20260928 23:07:06.660308 22957 webserver.cc:460] Webserver started at http://127.15.212.124:41251/ using document root <none> and password file <none>
I20260928 23:07:06.660425 22957 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.660483 22957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.661113 22957 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:06.661635 23071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.661798 22957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.661842 22957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "3773c0660e474c498d93334f2113cc3f"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a33cb16646778538cd3639c2fb7c0326"
server_key_iv: "d2f8eb988513ec8fc98f862c95a54865"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.661901 22957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.670059 22957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.671712 22957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.674753 22957 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.124:46229
I20260928 23:07:06.675050 22957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
I20260928 23:07:06.675266 23123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.124:46229 every 8 connection(s)
I20260928 23:07:06.675890 23124 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:07:06.676285 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Bootstrap starting.
I20260928 23:07:06.678594 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:07:06.678821 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Bootstrap complete.
I20260928 23:07:06.679013 23124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } }
I20260928 23:07:06.679100 23124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3773c0660e474c498d93334f2113cc3f, State: Initialized, Role: LEARNER
I20260928 23:07:06.679143 23124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } }
I20260928 23:07:06.679335 23124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:06.679311 23127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:06.679502 23127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: LEARNER
I20260928 23:07:06.680958 23138 catalog_manager.cc:1259] Loaded cluster ID: 469391a168a9412d9e54d64e34b39cdb
I20260928 23:07:06.681037 23138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: loading cluster ID for follower catalog manager: success
I20260928 23:07:06.682097 23138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: acquiring CA information for follower catalog manager: success
I20260928 23:07:06.682380 23138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 23:07:06.728834 23050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.729059 23050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.729100 23050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.730433 23050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.730491 23050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.730568 23050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:06.732319 23050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:35637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=41425
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:06.732592 23050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:06.732829 23050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.738631 23143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.738638 23144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.738701 23146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.738933 23050 server_base.cc:1034] running on GCE node
I20260928 23:07:06.739148 23050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.739374 23050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.740526 23050 hybrid_clock.cc:648] HybridClock initialized: now 1790636826740504 us; error 28 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.742933 23050 init.cc:377] Logged in from keytab as oryx/127.15.212.65@KRBTEST.COM (short username oryx)
I20260928 23:07:06.743377 23050 webserver.cc:460] Webserver started at http://127.15.212.65:41425/ using document root <none> and password file <none>
I20260928 23:07:06.743597 23050 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.743680 23050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.745046 23050 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:07:06.745589 23153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.745750 23050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:06.745851 23050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "dd33f746945d48b3b9737ae7a52f4716"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "70f5dde20319f304bb41df1950690275"
server_key_iv: "26dd811263b247249fa9384414840820"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.746182 23050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.760236 23050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.762609 23050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:06.762774 23050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.762987 23050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:06.763288 23050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:06.763320 23050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.763372 23050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:06.763399 23050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.768922 23050 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:35637
I20260928 23:07:06.769032 23266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:35637 every 8 connection(s)
I20260928 23:07:06.769330 23050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:06.776883 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23050
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:06.777040 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22499
I20260928 23:07:06.782737 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:38407 (local address 127.15.212.126:41451)
0928 23:07:06.770266 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.770268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.770505 (+ 237us) server_negotiation.cc:407] Connection header received
0928 23:07:06.770931 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.770933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.770945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.771005 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.771926 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772529 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.773940 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774191 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774261 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.776878 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.776880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.776880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.776887 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.779011 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.781606 (+ 2595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.781608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.781608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.781687 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.781988 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.781989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.781990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.782132 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.782178 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.782553 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.782647 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 23:07:06.783134 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:38407)
0928 23:07:06.770417 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.770434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.770841 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.771353 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.771354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.771600 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.771815 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.771819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772610 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.772611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.773828 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.773830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774272 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.774336 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.774919 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.774923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.776786 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.779113 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.779118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.779121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.781507 (+ 2386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.781766 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.781767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.781767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.781907 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.782224 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.782229 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.782385 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.782858 (+ 473us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.782953 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":301,"thread_start_us":82,"threads_started":1}
I20260928 23:07:06.783206 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:45219
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:41451
--webserver_port=36441
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:07:06.783551 23267 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:06.783656 23267 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.783758 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:41407 (local address 127.15.212.124:46229)
0928 23:07:06.770295 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.770298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.770960 (+ 662us) server_negotiation.cc:407] Connection header received
0928 23:07:06.770987 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.770988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.771001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.771066 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.771923 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772886 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.773768 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774049 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774114 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.776469 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.776470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.776470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.776477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.779324 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.781869 (+ 2545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.781870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.781870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.781930 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.782114 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.782114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.782115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.782255 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.782321 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.783104 (+ 783us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.783587 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":48,"threads_started":1}
I20260928 23:07:06.783854 23267 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:06.784080 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.15.212.65:41407)
0928 23:07:06.770341 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.770352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.770830 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.771108 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.771112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.771618 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.771815 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.771821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772967 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.772968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.773669 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.773671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774135 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.774281 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.774870 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.774881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.776345 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.779408 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.779408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.779409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.781507 (+ 2098us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.782000 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.782001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.782001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.782049 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.782344 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.782345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.782385 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.782508 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.782663 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":41,"thread_start_us":110,"threads_started":1}
I20260928 23:07:06.784281 22303 ts_manager.cc:194] Re-registered known tserver with Master: dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637)
I20260928 23:07:06.784363 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:46799 (local address 127.15.212.125:40431)
0928 23:07:06.770088 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.770091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.770392 (+ 301us) server_negotiation.cc:407] Connection header received
0928 23:07:06.770949 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.770950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.770963 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.770996 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.771923 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772698 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.773915 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774223 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774303 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.781307 (+ 7004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.781308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.781309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.781316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.783111 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.783503 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.783505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.783505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.783560 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.783766 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.783767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.783767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.783913 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.783953 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.784141 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.784222 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:07:06.784704 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.770027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.15.212.65:46799)
0928 23:07:06.770263 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.770306 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.770830 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.771108 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.771112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.771628 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.771815 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.771821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.772776 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.772776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.773830 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.773831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.774353 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.774354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.774411 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.775090 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.775095 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.781209 (+ 6114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.783203 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.783204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.783204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.783420 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.783628 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.783629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.783629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.783687 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.783995 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.783995 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.784036 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.784497 (+ 461us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.784609 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":33,"thread_start_us":92,"threads_started":1}
I20260928 23:07:06.784857 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:38407
I20260928 23:07:06.785036 23269 heartbeater.cc:347] Connected to a master server at 127.15.212.125:40431
I20260928 23:07:06.785084 23269 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.785144 23269 heartbeater.cc:511] Master 127.15.212.125:40431 requested a full tablet report, sending...
I20260928 23:07:06.785178 23268 heartbeater.cc:347] Connected to a master server at 127.15.212.124:46229
I20260928 23:07:06.785220 23268 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.785274 23268 heartbeater.cc:511] Master 127.15.212.124:46229 requested a full tablet report, sending...
I20260928 23:07:06.785513 22903 ts_manager.cc:194] Registered new tserver with Master: dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637)
I20260928 23:07:06.785597 23088 ts_manager.cc:194] Registered new tserver with Master: dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637)
W20260928 23:07:06.866426 23275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.866676 23275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.866699 23275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.867781 23275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.867815 23275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.867841 23275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:06.869277 23275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:45219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=36441
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:06.869449 23275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:06.869621 23275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:06.874603 23281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:06.874650 23280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.874820 23275 server_base.cc:1034] running on GCE node
W20260928 23:07:06.874608 23283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:06.875159 23275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:06.875397 23275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:06.876569 23275 hybrid_clock.cc:648] HybridClock initialized: now 1790636826876542 us; error 35 us; skew 500 ppm
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:06.879077 23275 init.cc:377] Logged in from keytab as oryx/127.15.212.66@KRBTEST.COM (short username oryx)
I20260928 23:07:06.879562 23275 webserver.cc:460] Webserver started at http://127.15.212.66:36441/ using document root <none> and password file <none>
I20260928 23:07:06.879799 23275 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:06.879853 23275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:06.881443 23275 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:06.882210 23290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.882357 23275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:06.882441 23275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "c0e2059b4fee46f296d35384db3e6a75"
format_stamp: "Formatted at 2026-09-28 23:07:05 on dist-test-slave-x2t8"
server_key: "e90b7d5eabe6992522612fc99c34b9ba"
server_key_iv: "bbdce1e6140242fc5cb52e19fcf7d856"
server_key_version: "encryptionkey@0"
I20260928 23:07:06.882752 23275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:06.893426 23275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:06.895606 23275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:06.895722 23275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:06.895895 23275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:06.896127 23275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:06.896158 23275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.896179 23275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:06.896193 23275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:06.901897 23275 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:45219
I20260928 23:07:06.901978 23403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:45219 every 8 connection(s)
I20260928 23:07:06.902292 23275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:06.909164 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23275
I20260928 23:07:06.909333 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22633
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
Sep 28 23:07:06 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636826, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
I20260928 23:07:06.912967 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.903040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:41389 (local address 127.15.212.125:40431)
0928 23:07:06.903204 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.903207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.903550 (+ 343us) server_negotiation.cc:407] Connection header received
0928 23:07:06.904399 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.904400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.904412 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.904526 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.905288 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.905771 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.906779 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.906966 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907039 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.909480 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.909481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.909481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.909488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.911241 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.911742 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.911744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.911744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.911806 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.912116 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.912117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.912117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.912270 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.912320 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.912772 (+ 452us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.912848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260928 23:07:06.913582 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.902969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.15.212.66:41389)
0928 23:07:06.903479 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.903488 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.904309 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.904622 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.904623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.904995 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.905185 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.905190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.905874 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.905875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.906677 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.906679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.907043 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.907206 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.907753 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.907764 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.909367 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.911325 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.911331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.911333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.911670 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.911882 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.911883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.911884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.912025 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.912376 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.912382 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.912599 (+ 217us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.913112 (+ 513us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.913321 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260928 23:07:06.914033 23406 heartbeater.cc:347] Connected to a master server at 127.15.212.125:40431
I20260928 23:07:06.914135 23406 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.914330 23406 heartbeater.cc:511] Master 127.15.212.125:40431 requested a full tablet report, sending...
I20260928 23:07:06.914350 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.903207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:40147 (local address 127.15.212.126:41451)
0928 23:07:06.903269 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.903271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.903354 (+ 83us) server_negotiation.cc:407] Connection header received
0928 23:07:06.904424 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.904426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.904456 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.904490 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.905299 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.906065 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907194 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.907431 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907501 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.911133 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.911135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.911135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.911144 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.913123 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913517 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.913519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.913519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.913576 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913791 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.913792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.913792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.913928 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.913970 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.914189 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.914264 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 23:07:06.914532 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.903093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:37487 (local address 127.15.212.124:46229)
0928 23:07:06.903276 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:06.903278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:06.903395 (+ 117us) server_negotiation.cc:407] Connection header received
0928 23:07:06.904422 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:06.904423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:06.904456 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:06.904490 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:06.905299 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.906063 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907164 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.907460 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907544 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.910800 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:06.910801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:06.910802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:06.910810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:06.912645 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913329 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.913331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.913331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.913392 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913616 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:06.913617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:06.913617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:06.913755 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:06.913795 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:06.914303 (+ 508us) server_negotiation.cc:299] Negotiation successful
0928 23:07:06.914395 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260928 23:07:06.914631 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.902970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:40147)
0928 23:07:06.903225 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.903270 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.904301 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.904616 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.904620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.905028 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.905186 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.905191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.906166 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.906167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.907087 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.907089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.907509 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.907575 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.907924 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.907928 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.911018 (+ 3090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.913213 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.913214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.913433 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.913654 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.913655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.913716 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.914017 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.914018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.914058 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.914424 (+ 366us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.914516 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":49,"spinlock_wait_cycles":6912,"thread_start_us":99,"threads_started":1}
I20260928 23:07:06.914759 22903 ts_manager.cc:194] Registered new tserver with Master: c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:06.915591 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:06.902970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.15.212.66:37487)
0928 23:07:06.903263 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:06.903272 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:06.904298 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:06.904616 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:06.904620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:06.904997 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:06.905186 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.905191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.906166 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.906167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.907079 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:06.907082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:06.907596 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:06.907597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:06.907708 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:06.908124 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:06.908136 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:06.910708 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:06.912734 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.912735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.912735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.912954 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.913485 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:06.913486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:06.913486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:06.913549 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:06.914061 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:06.914062 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:06.914154 (+ 92us) client_negotiation.cc:769] Sending connection context
0928 23:07:06.915123 (+ 969us) client_negotiation.cc:240] Negotiation successful
0928 23:07:06.915245 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":46,"thread_start_us":100,"threads_started":1}
I20260928 23:07:06.916026 23405 heartbeater.cc:347] Connected to a master server at 127.15.212.124:46229
I20260928 23:07:06.916088 23405 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.916163 23405 heartbeater.cc:511] Master 127.15.212.124:46229 requested a full tablet report, sending...
I20260928 23:07:06.916481 23088 ts_manager.cc:194] Registered new tserver with Master: c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:06.916635 23404 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:06.916690 23404 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:06.916756 23404 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:06.916990 22303 ts_manager.cc:194] Re-registered known tserver with Master: c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:06.917356 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:40147
I20260928 23:07:06.917733 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:37281
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:41451
--webserver_port=45147
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:06.998044 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:06.998242 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:06.998265 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:06.999342 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:06.999378 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:06.999403 23411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:07.000798 23411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:37281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=45147
--tserver_master_addrs=127.15.212.126:41451,127.15.212.125:40431,127.15.212.124:46229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:07.000983 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:07.001155 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:07.005903 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:07.005975 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:07.005894 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:07.006109 23411 server_base.cc:1034] running on GCE node
I20260928 23:07:07.006304 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:07.006541 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:07.007674 23411 hybrid_clock.cc:648] HybridClock initialized: now 1790636827007657 us; error 38 us; skew 500 ppm
Sep 28 23:07:07 dist-test-slave-x2t8 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636827, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:07.010090 23411 init.cc:377] Logged in from keytab as oryx/127.15.212.67@KRBTEST.COM (short username oryx)
I20260928 23:07:07.010447 23411 webserver.cc:460] Webserver started at http://127.15.212.67:45147/ using document root <none> and password file <none>
I20260928 23:07:07.010628 23411 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:07.010676 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:07.011897 23411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:07.012398 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:07.012583 23411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:07.012673 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "38049d9a357149309c1e5891272a6c2b"
format_stamp: "Formatted at 2026-09-28 23:07:06 on dist-test-slave-x2t8"
server_key: "a25bf188fe346851be49c1c342b3a5f9"
server_key_iv: "8ec0614788179aa52a61a8eef0796415"
server_key_version: "encryptionkey@0"
I20260928 23:07:07.012965 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:07.029160 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:07.030857 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:07.030962 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:07.031179 23411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:07.031487 23411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:07.031541 23411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:07.031582 23411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:07.031608 23411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:07.037077 23411 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:37281
I20260928 23:07:07.037149 23539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:37281 every 8 connection(s)
I20260928 23:07:07.037489 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:07.042182 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23411
Sep 28 23:07:07 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636827, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
Sep 28 23:07:07 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636827, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.124@KRBTEST.COM
Sep 28 23:07:07 dist-test-slave-x2t8 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636827, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.125@KRBTEST.COM
I20260928 23:07:07.049928 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.15.212.67:32807)
0928 23:07:07.038499 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.038514 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.039018 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.039268 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.039269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.039542 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:07.039707 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.039713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040476 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.040477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.041566 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.041567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.042442 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.042444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.042511 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.042862 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:07.042868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:07.045678 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:07.048094 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.048101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.048103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.048465 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.048706 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.048707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.048707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.048848 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.049156 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:07.049162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:07.049354 (+ 192us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.049477 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.049684 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260928 23:07:07.050400 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:32807 (local address 127.15.212.124:46229)
0928 23:07:07.038279 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.038281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.038598 (+ 317us) server_negotiation.cc:407] Connection header received
0928 23:07:07.039119 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.039120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.039133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.039166 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.039819 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040340 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.042046 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.042356 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.042441 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.045789 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.045790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.045791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.045798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.048002 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.048568 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.048569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.048570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.048631 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.048926 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.048927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.048927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.049063 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.049127 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.050214 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.050300 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:07:07.050770 23541 heartbeater.cc:347] Connected to a master server at 127.15.212.124:46229
I20260928 23:07:07.050877 23541 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:07.051079 23541 heartbeater.cc:511] Master 127.15.212.124:46229 requested a full tablet report, sending...
I20260928 23:07:07.051510 23088 ts_manager.cc:194] Registered new tserver with Master: 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:07.054085 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.044102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.15.212.124:46229)
0928 23:07:07.046341 (+ 2239us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.046343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.046350 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:07.046369 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.046369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.046380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.047137 (+ 757us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.047170 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.047871 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.048982 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.049297 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.049592 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.050683 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.050684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.050685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.050691 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.052581 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.053009 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.053010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.053010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.053065 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.053327 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.053328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.053328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.053466 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.053864 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.053903 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.053975 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2218,"thread_start_us":62,"threads_started":1}
I20260928 23:07:07.054482 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.043735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.15.212.125:40431)
0928 23:07:07.044345 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.044347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.044354 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:07.044371 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.044372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.044382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.044489 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.045437 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.046198 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.047422 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.047721 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.047757 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.049436 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.049438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.049438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.049446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.053148 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.053510 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.053512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.053512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.053568 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.053869 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.053870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.053870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.054011 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.054079 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.054310 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.054388 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":56,"threads_started":1}
I20260928 23:07:07.054993 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:42029 (local address 127.15.212.126:41451)
0928 23:07:07.038303 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.038305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.038598 (+ 293us) server_negotiation.cc:407] Connection header received
0928 23:07:07.039119 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.039121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.039133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.039166 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.039918 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040662 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.041501 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.041786 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.041853 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.045000 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.045001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.045002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.045010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.047307 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.052295 (+ 4988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.052296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.052296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.052374 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.052646 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.052647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.052647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.052814 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.052869 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.054798 (+ 1929us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.054879 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:07:07.055132 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:60383 (local address 127.15.212.125:40431)
0928 23:07:07.038240 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.038242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.038598 (+ 356us) server_negotiation.cc:407] Connection header received
0928 23:07:07.039092 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.039093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.039105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.039169 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.039796 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040601 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.041644 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.041922 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.041978 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.051947 (+ 9969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.051948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.051949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.051956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.053795 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.054269 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.054270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.054270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.054326 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.054611 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.054612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.054612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.054751 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.054810 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.054981 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.055053 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 23:07:07.055294 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:42029)
0928 23:07:07.038524 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.038532 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.039018 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.039626 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.039627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.039790 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:07.039844 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.039846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040723 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.040724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.041399 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.041401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.041862 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.041863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.041975 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.042682 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:07.042706 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:07.044891 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:07.051980 (+ 7089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.051980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.051980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.052208 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.052496 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.052497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.052497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.052564 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.054635 (+ 2071us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:07.054636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:07.054677 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.055073 (+ 396us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.055171 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260928 23:07:07.055411 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.038197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.15.212.67:60383)
0928 23:07:07.038470 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.038511 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.038995 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.039243 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.039246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.039522 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:07.039707 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.039710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.040675 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.040675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.041560 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.041562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.042010 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.042010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.042065 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.042899 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:07.042903 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:07.051881 (+ 8978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:07.053886 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.053887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.053887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.054085 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.054450 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:07.054451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:07.054451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:07.054510 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:07.054836 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:07.054837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:07.054874 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.055203 (+ 329us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.055279 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":6,"thread_start_us":78,"threads_started":1}
I20260928 23:07:07.055621 23540 heartbeater.cc:347] Connected to a master server at 127.15.212.126:41451
I20260928 23:07:07.055658 23540 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:07.055723 23540 heartbeater.cc:511] Master 127.15.212.126:41451 requested a full tablet report, sending...
I20260928 23:07:07.056007 22303 ts_manager.cc:194] Re-registered known tserver with Master: 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:07.056538 22303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:42029
I20260928 23:07:07.056649 23542 heartbeater.cc:347] Connected to a master server at 127.15.212.125:40431
I20260928 23:07:07.056697 23542 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:07.056762 23542 heartbeater.cc:511] Master 127.15.212.125:40431 requested a full tablet report, sending...
I20260928 23:07:07.057013 22903 ts_manager.cc:194] Registered new tserver with Master: 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:07.057201 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.043772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.15.212.126:41451)
0928 23:07:07.044565 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.044567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.044574 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:07.044596 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.044597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.044609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.044688 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.045496 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.046230 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.047663 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.048008 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.048352 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.050304 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.050305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.050305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.050312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.053941 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.056221 (+ 2280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.056222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.056222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.056274 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.056501 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.056502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.056503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.056744 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.056844 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.057006 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.057087 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":76,"threads_started":1}
I20260928 23:07:07.058490 22303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:07.059967 22303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:07.067346 22862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.063785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:35637 (local address 127.0.0.1:43936)
0928 23:07:07.063841 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.063853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.063891 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.064249 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.064250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.064252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.064290 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.064291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065578 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.065579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.066441 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.066443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.067116 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.067117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.067175 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.067179 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.067228 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.067265 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 23:07:07.067634 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.063877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.15.212.65:35637)
0928 23:07:07.064057 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.064061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.064072 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:07:07.064126 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.064129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.064146 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.064247 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.064377 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065479 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.066531 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.067017 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.067110 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.067472 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.067503 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.067547 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260928 23:07:07.068343 23201 tablet_service.cc:1467] Processing CreateTablet for tablet ce62ba8bb989441a95477263407e1044 (DEFAULT_TABLE table=test-table [id=33ae1e0127a54f25802b34a2f19f7ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:07.068511 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.063981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:45219 (local address 127.0.0.1:34664)
0928 23:07:07.064337 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.064346 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.064393 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.064860 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.064861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.064863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.064891 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.064892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065804 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.065805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.067015 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.067017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.068230 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.068231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.068276 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.068281 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.068366 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.068407 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":32,"thread_start_us":77,"threads_started":1}
I20260928 23:07:07.068522 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.064076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.15.212.66:45219)
0928 23:07:07.064227 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.064232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.064727 (+ 495us) server_negotiation.cc:407] Connection header received
0928 23:07:07.064768 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.064771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.064786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.064864 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.064943 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065719 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.067470 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.068148 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.068223 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.068292 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.068386 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.068427 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260928 23:07:07.068778 23201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce62ba8bb989441a95477263407e1044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:07.069610 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.063791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:37281 (local address 127.0.0.1:34450)
0928 23:07:07.064237 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.064247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.064469 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.064676 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.064677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.064679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.064721 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.064723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065849 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.065850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.067019 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.067021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.069452 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.069452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.069498 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.069500 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.069540 (+ 40us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.069569 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":55,"thread_start_us":86,"threads_started":1}
I20260928 23:07:07.069895 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.063875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.15.212.67:37281)
0928 23:07:07.064546 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.064550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.064559 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:07.064592 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.064594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.064608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.064680 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.064798 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.065752 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.068326 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.069374 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.069452 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.069518 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.069549 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.069583 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":91,"threads_started":1}
I20260928 23:07:07.070463 23474 tablet_service.cc:1467] Processing CreateTablet for tablet ce62ba8bb989441a95477263407e1044 (DEFAULT_TABLE table=test-table [id=33ae1e0127a54f25802b34a2f19f7ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:07.070736 23474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce62ba8bb989441a95477263407e1044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:07.072039 23566 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Bootstrap starting.
I20260928 23:07:07.072732 23566 tablet_bootstrap.cc:654] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:07.072624 23338 tablet_service.cc:1467] Processing CreateTablet for tablet ce62ba8bb989441a95477263407e1044 (DEFAULT_TABLE table=test-table [id=33ae1e0127a54f25802b34a2f19f7ec0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:07.072960 23338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce62ba8bb989441a95477263407e1044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:07.073040 23566 log.cc:824] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:07.073774 23568 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Bootstrap starting.
I20260928 23:07:07.073829 23566 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: No bootstrap required, opened a new log
I20260928 23:07:07.073895 23566 ts_tablet_manager.cc:1397] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:07.074398 23568 tablet_bootstrap.cc:654] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:07.074678 23568 log.cc:824] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:07.075888 23566 raft_consensus.cc:348] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.076113 23566 raft_consensus.cc:374] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:07.076157 23566 raft_consensus.cc:729] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd33f746945d48b3b9737ae7a52f4716, State: Initialized, Role: FOLLOWER
I20260928 23:07:07.076244 23570 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Bootstrap starting.
I20260928 23:07:07.076296 23566 consensus_queue.cc:260] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.076599 23566 ts_tablet_manager.cc:1428] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 23:07:07.076735 23267 heartbeater.cc:503] Master 127.15.212.126:41451 was elected leader, sending a full tablet report...
I20260928 23:07:07.076881 23570 tablet_bootstrap.cc:654] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:07.076960 23568 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: No bootstrap required, opened a new log
I20260928 23:07:07.077020 23568 ts_tablet_manager.cc:1397] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 23:07:07.077184 23570 log.cc:824] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:07.078382 23570 tablet_bootstrap.cc:492] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: No bootstrap required, opened a new log
I20260928 23:07:07.078464 23570 ts_tablet_manager.cc:1397] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:07.078904 23568 raft_consensus.cc:348] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.079017 23568 raft_consensus.cc:374] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:07.079038 23568 raft_consensus.cc:729] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38049d9a357149309c1e5891272a6c2b, State: Initialized, Role: FOLLOWER
I20260928 23:07:07.079273 23568 consensus_queue.cc:260] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.079571 23568 ts_tablet_manager.cc:1428] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:07:07.079576 23540 heartbeater.cc:503] Master 127.15.212.126:41451 was elected leader, sending a full tablet report...
I20260928 23:07:07.081329 23570 raft_consensus.cc:348] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.081499 23570 raft_consensus.cc:374] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:07.081532 23570 raft_consensus.cc:729] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e2059b4fee46f296d35384db3e6a75, State: Initialized, Role: FOLLOWER
I20260928 23:07:07.081660 23570 consensus_queue.cc:260] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.081904 23570 ts_tablet_manager.cc:1428] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260928 23:07:07.081956 23404 heartbeater.cc:503] Master 127.15.212.126:41451 was elected leader, sending a full tablet report...
W20260928 23:07:07.153650 23407 tablet.cc:2369] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:07.251998 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.249283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:39306)
0928 23:07:07.249347 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.249359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.249410 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.249606 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.249606 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.249608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.249674 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.249676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.250335 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.250337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.251059 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.251061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.251751 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.251752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.251813 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.251817 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.251868 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.251920 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 23:07:07.251998 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.249344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.15.212.124:46229)
0928 23:07:07.249398 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.249400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.249410 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:07.249482 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.249483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.249511 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.249587 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.249751 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.250244 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.251160 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.251644 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.251751 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.251836 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.251885 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.251923 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 23:07:07.252564 23091 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260928 23:07:07.253571 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65ebb053fe164ab4ae2b365316d730ff. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:07.253691 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: LEARNER
W20260928 23:07:07.270164 23270 tablet.cc:2369] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:07:07.289255 23543 tablet.cc:2369] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:07.299039 23573 raft_consensus.cc:482] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:07.299154 23573 raft_consensus.cc:504] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.299566 23573 leader_election.cc:288] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637), c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:07.303426 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.299844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48653 (local address 127.15.212.66:45219)
0928 23:07:07.299897 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.299899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.299906 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:07.299930 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.299930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.299940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.300021 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.300205 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.300971 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.302495 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.303053 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.303127 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.303135 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.303315 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.303356 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:07:07.303426 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.299775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:45567 (local address 127.15.212.65:35637)
0928 23:07:07.299843 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.299846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.299858 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:07:07.299930 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.299931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.299940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.300004 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:07.300204 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.300996 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.302500 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.303046 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.303127 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.303135 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.303315 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.303361 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:07:07.303447 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.299758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:45219 (local address 127.15.212.67:48653)
0928 23:07:07.299811 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.299824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.299866 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.300021 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.300021 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.300022 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.300129 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.300131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.301069 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.301070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.302393 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.302395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.303141 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.303142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.303229 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.303235 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.303297 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.303339 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":91}
I20260928 23:07:07.303460 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.299745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:35637 (local address 127.15.212.67:45567)
0928 23:07:07.299811 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:07.299824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:07.299866 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:07.300019 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:07.300020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:07.300022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:07.300129 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.300131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.301069 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.301070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.302393 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:07.302395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.303141 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.303142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:07.303229 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.303235 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:07:07.303297 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:07:07.303339 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":112}
I20260928 23:07:07.303902 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce62ba8bb989441a95477263407e1044" candidate_uuid: "38049d9a357149309c1e5891272a6c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e2059b4fee46f296d35384db3e6a75" is_pre_election: true
I20260928 23:07:07.304080 23358 raft_consensus.cc:2454] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38049d9a357149309c1e5891272a6c2b in term 0.
I20260928 23:07:07.304271 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce62ba8bb989441a95477263407e1044" candidate_uuid: "38049d9a357149309c1e5891272a6c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd33f746945d48b3b9737ae7a52f4716" is_pre_election: true
I20260928 23:07:07.304304 23429 leader_election.cc:302] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38049d9a357149309c1e5891272a6c2b, c0e2059b4fee46f296d35384db3e6a75; no voters:
I20260928 23:07:07.304386 23221 raft_consensus.cc:2454] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38049d9a357149309c1e5891272a6c2b in term 0.
I20260928 23:07:07.304467 23573 raft_consensus.cc:2781] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:07.304559 23573 raft_consensus.cc:482] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:07.304597 23573 raft_consensus.cc:3037] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:07.305294 23573 raft_consensus.cc:504] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.305439 23573 leader_election.cc:288] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [CANDIDATE]: Term 1 election: Requested vote from peers dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637), c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219)
I20260928 23:07:07.305606 23358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce62ba8bb989441a95477263407e1044" candidate_uuid: "38049d9a357149309c1e5891272a6c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0e2059b4fee46f296d35384db3e6a75"
I20260928 23:07:07.305606 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce62ba8bb989441a95477263407e1044" candidate_uuid: "38049d9a357149309c1e5891272a6c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd33f746945d48b3b9737ae7a52f4716"
I20260928 23:07:07.305680 23221 raft_consensus.cc:3037] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:07.305680 23358 raft_consensus.cc:3037] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:07.306496 23221 raft_consensus.cc:2454] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38049d9a357149309c1e5891272a6c2b in term 1.
I20260928 23:07:07.306496 23358 raft_consensus.cc:2454] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38049d9a357149309c1e5891272a6c2b in term 1.
I20260928 23:07:07.306707 23428 leader_election.cc:302] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38049d9a357149309c1e5891272a6c2b, dd33f746945d48b3b9737ae7a52f4716; no voters:
I20260928 23:07:07.306809 23573 raft_consensus.cc:2781] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:07.306969 23573 raft_consensus.cc:686] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 1 LEADER]: Becoming Leader. State: Replica: 38049d9a357149309c1e5891272a6c2b, State: Running, Role: LEADER
I20260928 23:07:07.307183 23573 consensus_queue.cc:237] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } }
I20260928 23:07:07.308323 22303 catalog_manager.cc:5495] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 38049d9a357149309c1e5891272a6c2b (127.15.212.67). New cstate: current_term: 1 leader_uuid: "38049d9a357149309c1e5891272a6c2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } health_report { overall_health: UNKNOWN } } }
I20260928 23:07:07.308957 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: NON_VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: true } } }
I20260928 23:07:07.309079 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:07.309664 22864 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: attempting to promote NON_VOTER 3773c0660e474c498d93334f2113cc3f to VOTER
I20260928 23:07:07.309795 22864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:07.310257 22913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Refusing update from remote peer 65ebb053fe164ab4ae2b365316d730ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 23:07:07.310380 23098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 LEARNER]: Refusing update from remote peer 65ebb053fe164ab4ae2b365316d730ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 23:07:07.310565 23582 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260928 23:07:07.310671 22863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260928 23:07:07.311355 22864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3773c0660e474c498d93334f2113cc3f (127.15.212.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.311460 22913 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3773c0660e474c498d93334f2113cc3f (127.15.212.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.311515 23091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 3773c0660e474c498d93334f2113cc3f (127.15.212.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312204 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312286 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:07.312337 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312314 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312388 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:07.312405 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:07.312477 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312553 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:07.312656 23584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65ebb053fe164ab4ae2b365316d730ff" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:07.312747 23584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:07.348982 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.346529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34454 (local address 127.15.212.67:37281)
0928 23:07:07.346599 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.346600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.346607 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:07.346655 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.346656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.346674 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.346750 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:07.346905 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.347375 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.348121 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.348333 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.348420 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.348650 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:07.348763 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:07.348841 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.348890 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.348909 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:07:07.350790 23358 raft_consensus.cc:1261] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Refusing update from remote peer 38049d9a357149309c1e5891272a6c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:07.350790 23221 raft_consensus.cc:1261] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Refusing update from remote peer 38049d9a357149309c1e5891272a6c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:07.351127 23573 consensus_queue.cc:1035] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:07.351302 23580 consensus_queue.cc:1035] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:07.353649 23586 mvcc.cc:205] Tried to move back new op lower bound from 7334448444826013696 to 7334448444650778624. Current Snapshot: MvccSnapshot[applied={T|T < 7334448444826013696}]
I20260928 23:07:07.354534 23588 mvcc.cc:205] Tried to move back new op lower bound from 7334448444826013696 to 7334448444650778624. Current Snapshot: MvccSnapshot[applied={T|T < 7334448444826013696 or (T in {7334448444826013696})}]
I20260928 23:07:07.357621 22302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:07.357717 22302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:07.359006 22302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65ebb053fe164ab4ae2b365316d730ff: Sending DeleteTablet for 3 replicas of tablet ce62ba8bb989441a95477263407e1044
I20260928 23:07:07.359360 23474 tablet_service.cc:1514] Processing DeleteTablet for tablet ce62ba8bb989441a95477263407e1044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:07 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:34450
I20260928 23:07:07.359356 23201 tablet_service.cc:1514] Processing DeleteTablet for tablet ce62ba8bb989441a95477263407e1044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:07 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:43936
I20260928 23:07:07.359543 23596 tablet_replica.cc:330] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: stopping tablet replica
I20260928 23:07:07.359539 23338 tablet_service.cc:1514] Processing DeleteTablet for tablet ce62ba8bb989441a95477263407e1044 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:07 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:34664
I20260928 23:07:07.359624 23596 raft_consensus.cc:2229] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:07.359742 23596 raft_consensus.cc:2258] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:07.360049 23595 tablet_replica.cc:330] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: stopping tablet replica
I20260928 23:07:07.360143 23595 raft_consensus.cc:2229] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:07.360220 23596 ts_tablet_manager.cc:1905] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:07.360249 23595 raft_consensus.cc:2258] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:07.360384 23597 tablet_replica.cc:330] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: stopping tablet replica
I20260928 23:07:07.360867 23597 raft_consensus.cc:2229] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:07.360977 23597 raft_consensus.cc:2258] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:07.361027 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22271
I20260928 23:07:07.362681 23596 ts_tablet_manager.cc:1918] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:07.362749 23596 log.cc:1196] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/ce62ba8bb989441a95477263407e1044
I20260928 23:07:07.362779 23597 ts_tablet_manager.cc:1905] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:07.363004 23596 ts_tablet_manager.cc:1939] T ce62ba8bb989441a95477263407e1044 P 38049d9a357149309c1e5891272a6c2b: Deleting consensus metadata
I20260928 23:07:07.364632 23597 ts_tablet_manager.cc:1918] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:07.364705 23597 log.cc:1196] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/ce62ba8bb989441a95477263407e1044
I20260928 23:07:07.364951 23597 ts_tablet_manager.cc:1939] T ce62ba8bb989441a95477263407e1044 P c0e2059b4fee46f296d35384db3e6a75: Deleting consensus metadata
W20260928 23:07:07.367240 23292 connection.cc:535] server connection from 127.0.0.1:34664 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 23:07:07.367427 23430 connection.cc:535] server connection from 127.0.0.1:34450 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:07:07.368481 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.359880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.15.212.124:46229)
0928 23:07:07.359917 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.359919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.360721 (+ 802us) server_negotiation.cc:407] Connection header received
0928 23:07:07.361110 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.361111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.361122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.361150 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.362508 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.363337 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.364039 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.364248 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.364327 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.365799 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.365800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.365801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.365808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.367315 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.367695 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.367696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.367697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.367750 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.367949 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.367950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.367950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.368063 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.368181 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.368281 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.368358 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:07:07.369422 23595 ts_tablet_manager.cc:1905] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:07.371007 23595 ts_tablet_manager.cc:1918] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:07.371078 23595 log.cc:1196] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/ce62ba8bb989441a95477263407e1044
I20260928 23:07:07.371327 23595 ts_tablet_manager.cc:1939] T ce62ba8bb989441a95477263407e1044 P dd33f746945d48b3b9737ae7a52f4716: Deleting consensus metadata
W20260928 23:07:07.371650 23154 connection.cc:441] server connection from 127.0.0.1:43936 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43936 (request call id 1) could send its response
I20260928 23:07:07.376567 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.369595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.15.212.124:46229)
0928 23:07:07.369641 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.369642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.369781 (+ 139us) server_negotiation.cc:407] Connection header received
0928 23:07:07.369873 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.369874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.369883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.369912 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.370556 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.371245 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.372312 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.372580 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.372651 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.373922 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.373923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.373924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.373930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.375396 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.375750 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.375751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.375751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.375801 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.376001 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.376002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.376002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.376112 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.376218 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.376368 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.376476 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:07:07.376885 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:07.369276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.15.212.125:40431)
0928 23:07:07.369316 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:07.369318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:07.369479 (+ 161us) server_negotiation.cc:407] Connection header received
0928 23:07:07.369577 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:07.369578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:07.369590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:07.369620 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:07.370356 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.371108 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.372138 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:07.372490 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:07.372562 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:07.374017 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:07.374018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:07.374018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:07.374025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:07.375666 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.376055 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.376056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.376056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.376108 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:07.376285 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:07.376287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:07.376287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:07.376421 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:07.376533 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:07.376721 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 23:07:07.376792 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:07:08.355343 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.355048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:37731)
0928 23:07:08.355167 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.355283 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
W20260928 23:07:08.355681 23404 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:41451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.356037 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.355825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:43477)
0928 23:07:08.355937 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.355997 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260928 23:07:08.356217 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.356013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:51095)
0928 23:07:08.356126 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.356182 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":49,"threads_started":1}
W20260928 23:07:08.356218 23540 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:41451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
W20260928 23:07:08.356427 23267 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:41451 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.859117 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65ebb053fe164ab4ae2b365316d730ff)
I20260928 23:07:08.859284 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:08.859582 23608 leader_election.cc:288] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125:40431)
I20260928 23:07:08.859609 23609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65ebb053fe164ab4ae2b365316d730ff)
I20260928 23:07:08.859733 23609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:08.859886 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.859687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35922)
0928 23:07:08.859800 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.859845 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":45,"threads_started":1}
I20260928 23:07:08.859973 23609 leader_election.cc:288] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 3773c0660e474c498d93334f2113cc3f (127.15.212.124:46229)
W20260928 23:07:08.859977 23073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.860363 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.860262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35946)
0928 23:07:08.860300 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.860332 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260928 23:07:08.860291 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.860029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35934)
0928 23:07:08.860167 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.860245 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":36,"threads_started":1}
W20260928 23:07:08.860502 23073 leader_election.cc:334] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
W20260928 23:07:08.860648 22888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.861069 23613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.860965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35956)
0928 23:07:08.861018 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.861048 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 23:07:08.861159 22888 leader_election.cc:334] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.863505 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.859836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.15.212.125:40431)
0928 23:07:08.859939 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:08.859941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:08.860273 (+ 332us) server_negotiation.cc:407] Connection header received
0928 23:07:08.860575 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:08.860576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:08.860586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:08.860624 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:08.860824 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.861541 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.862778 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.863230 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.863289 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:08.863296 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:08.863435 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 23:07:08.863462 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260928 23:07:08.863533 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.859749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:40431 (local address 127.0.0.1:46850)
0928 23:07:08.860141 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.860152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:08.860482 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:08.860712 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:08.860712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:08.860715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:08.860765 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:08.860767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.861620 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.861620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:08.862707 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:08.862709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.863303 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.863304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:08.863367 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:08.863370 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:08.863425 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 23:07:08.863465 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":1,"thread_start_us":41,"threads_started":1}
I20260928 23:07:08.863988 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.860341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.15.212.124:46229)
0928 23:07:08.860558 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:08.860560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:08.860567 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:08.860590 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:08.860591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:08.860601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:08.860680 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:08.861238 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.861887 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.863137 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.863665 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.863751 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:08.863758 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:08.863898 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 23:07:08.863932 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260928 23:07:08.863991 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.860143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:46229 (local address 127.0.0.1:57722)
0928 23:07:08.860310 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.860320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:08.860373 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:08.860682 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:08.860684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:08.860685 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:08.860765 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:08.860767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.861985 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.861986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:08.863041 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:08.863043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:08.863750 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:08.863751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:08.863814 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:08.863817 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:08.863889 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:07:08.863928 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":55,"threads_started":1}
I20260928 23:07:08.863988 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3773c0660e474c498d93334f2113cc3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" is_pre_election: true
I20260928 23:07:08.864151 22913 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3773c0660e474c498d93334f2113cc3f in term 1.
I20260928 23:07:08.864320 23074 leader_election.cc:302] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3773c0660e474c498d93334f2113cc3f, 8663db9cbc6c4d1b9c7070a81a99b9cc; no voters: 65ebb053fe164ab4ae2b365316d730ff
I20260928 23:07:08.864372 23091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3773c0660e474c498d93334f2113cc3f" is_pre_election: true
I20260928 23:07:08.864464 23608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 23:07:08.864529 23091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8663db9cbc6c4d1b9c7070a81a99b9cc in term 1.
I20260928 23:07:08.864568 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 65ebb053fe164ab4ae2b365316d730ff)
I20260928 23:07:08.864604 23608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:07:08.864704 22887 leader_election.cc:302] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3773c0660e474c498d93334f2113cc3f, 8663db9cbc6c4d1b9c7070a81a99b9cc; no voters: 65ebb053fe164ab4ae2b365316d730ff
I20260928 23:07:08.864806 23609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 23:07:08.864850 23609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 65ebb053fe164ab4ae2b365316d730ff)
I20260928 23:07:08.864868 23609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:07:08.865509 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:08.865667 23608 leader_election.cc:288] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 election: Requested vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125:40431)
I20260928 23:07:08.865665 23609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:08.865803 23609 leader_election.cc:288] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 election: Requested vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 3773c0660e474c498d93334f2113cc3f (127.15.212.124:46229)
I20260928 23:07:08.865844 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3773c0660e474c498d93334f2113cc3f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
I20260928 23:07:08.865913 22913 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3773c0660e474c498d93334f2113cc3f in current term 2: Already voted for candidate 8663db9cbc6c4d1b9c7070a81a99b9cc in this term.
I20260928 23:07:08.865911 23091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3773c0660e474c498d93334f2113cc3f"
I20260928 23:07:08.865984 23091 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8663db9cbc6c4d1b9c7070a81a99b9cc in current term 2: Already voted for candidate 3773c0660e474c498d93334f2113cc3f in this term.
I20260928 23:07:08.865988 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.865861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35978)
0928 23:07:08.865938 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.865960 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260928 23:07:08.866066 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.865794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35968)
0928 23:07:08.865881 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.865913 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 23:07:08.866328 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.866266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36002)
0928 23:07:08.866294 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.866315 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260928 23:07:08.866426 23073 leader_election.cc:334] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.866462 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:08.866266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:35986)
0928 23:07:08.866421 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:08.866444 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260928 23:07:08.866495 23073 leader_election.cc:302] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3773c0660e474c498d93334f2113cc3f; no voters: 65ebb053fe164ab4ae2b365316d730ff, 8663db9cbc6c4d1b9c7070a81a99b9cc
I20260928 23:07:08.866585 23608 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260928 23:07:08.866580 22888 leader_election.cc:334] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:08.866621 22888 leader_election.cc:302] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8663db9cbc6c4d1b9c7070a81a99b9cc; no voters: 3773c0660e474c498d93334f2113cc3f, 65ebb053fe164ab4ae2b365316d730ff
I20260928 23:07:08.866696 23609 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 23:07:09.356375 23618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:09.355996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:39829)
0928 23:07:09.356277 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:09.356320 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":82,"threads_started":1}
I20260928 23:07:09.356755 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:09.356491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:46967)
0928 23:07:09.356671 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:09.356706 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":58,"threads_started":1}
I20260928 23:07:09.356925 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:09.356708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:45241)
0928 23:07:09.356838 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:09.356880 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260928 23:07:10.357205 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.356856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:36869)
0928 23:07:10.357101 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.357146 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":143,"threads_started":1}
I20260928 23:07:10.357316 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.357106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:57111)
0928 23:07:10.357211 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.357250 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260928 23:07:10.357441 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.357281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:42185)
0928 23:07:10.357376 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.357404 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
I20260928 23:07:10.410382 23624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:10.410502 23624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:10.410722 23624 leader_election.cc:288] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125:40431)
I20260928 23:07:10.410957 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3773c0660e474c498d93334f2113cc3f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" is_pre_election: true
I20260928 23:07:10.411065 22913 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3773c0660e474c498d93334f2113cc3f in term 2.
I20260928 23:07:10.411206 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.410950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36196)
0928 23:07:10.411099 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.411141 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260928 23:07:10.411224 23074 leader_election.cc:302] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3773c0660e474c498d93334f2113cc3f, 8663db9cbc6c4d1b9c7070a81a99b9cc; no voters:
I20260928 23:07:10.411321 23624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260928 23:07:10.411355 23624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:10.411427 23624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 2 FOLLOWER]: Advancing to term 3
I20260928 23:07:10.411638 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.411546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36202)
0928 23:07:10.411582 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.411608 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260928 23:07:10.411723 23073 leader_election.cc:334] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:10.412233 23624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:10.412374 23624 leader_election.cc:288] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 election: Requested vote from peers 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451), 8663db9cbc6c4d1b9c7070a81a99b9cc (127.15.212.125:40431)
I20260928 23:07:10.412549 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.412465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36212)
0928 23:07:10.412502 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.412526 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260928 23:07:10.412561 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3773c0660e474c498d93334f2113cc3f" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc"
I20260928 23:07:10.412663 22913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 2 FOLLOWER]: Advancing to term 3
I20260928 23:07:10.412850 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.412769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36218)
0928 23:07:10.412809 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.412832 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 23:07:10.412941 23073 leader_election.cc:334] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
I20260928 23:07:10.413333 22913 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3773c0660e474c498d93334f2113cc3f in term 3.
I20260928 23:07:10.413565 23074 leader_election.cc:302] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3773c0660e474c498d93334f2113cc3f, 8663db9cbc6c4d1b9c7070a81a99b9cc; no voters: 65ebb053fe164ab4ae2b365316d730ff
I20260928 23:07:10.413698 23624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 3 FOLLOWER]: Leader election won for term 3
I20260928 23:07:10.413800 23624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [term 3 LEADER]: Becoming Leader. State: Replica: 3773c0660e474c498d93334f2113cc3f, State: Running, Role: LEADER
I20260928 23:07:10.413887 23624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } }
I20260928 23:07:10.414261 23627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3773c0660e474c498d93334f2113cc3f. Latest consensus state: current_term: 3 leader_uuid: "3773c0660e474c498d93334f2113cc3f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:10.414362 23627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:10.414542 23629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:10.415030 23629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:10.415108 23629 catalog_manager.cc:1259] Loaded cluster ID: 469391a168a9412d9e54d64e34b39cdb
I20260928 23:07:10.415143 23629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:10.415337 23629 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:10.415419 23629 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Loaded TSK: 0
I20260928 23:07:10.415582 23629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:10.453902 23088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:10.455444 23088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:10.456496 22913 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [term 3 FOLLOWER]: Refusing update from remote peer 3773c0660e474c498d93334f2113cc3f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260928 23:07:10.456543 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.456350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36232)
0928 23:07:10.456420 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.456477 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 23:07:10.456732 23627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260928 23:07:10.456883 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.456787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.0.0.1:36234)
0928 23:07:10.456836 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.456865 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:07:10.457054 23073 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f -> Peer 65ebb053fe164ab4ae2b365316d730ff (127.15.212.126:41451): Couldn't send request to peer 65ebb053fe164ab4ae2b365316d730ff. Status: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 23:07:10.457373 23631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3773c0660e474c498d93334f2113cc3f. Latest consensus state: current_term: 3 leader_uuid: "3773c0660e474c498d93334f2113cc3f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:10.457494 23631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:10.457938 23624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "3773c0660e474c498d93334f2113cc3f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:10.458056 23624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:10.458180 23631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "3773c0660e474c498d93334f2113cc3f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "65ebb053fe164ab4ae2b365316d730ff" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 41451 } } peers { permanent_uuid: "8663db9cbc6c4d1b9c7070a81a99b9cc" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 40431 } attrs { promote: false } } peers { permanent_uuid: "3773c0660e474c498d93334f2113cc3f" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 46229 } attrs { promote: false } } }
I20260928 23:07:10.458261 23631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8663db9cbc6c4d1b9c7070a81a99b9cc [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:10.463229 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:35637 (local address 127.0.0.1:53336)
0928 23:07:10.459783 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.459793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:10.459844 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:10.460250 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:10.460251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:10.460252 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:10.460306 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.460308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461248 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.461249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.462301 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.462303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.463016 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.463077 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463082 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:07:10.463112 (+ 30us) client_negotiation.cc:240] Negotiation successful
0928 23:07:10.463151 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":109}
I20260928 23:07:10.463474 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:45219 (local address 127.0.0.1:48866)
0928 23:07:10.460030 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.460040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:10.460084 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:10.460284 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:10.460284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:10.460286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:10.460310 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.460311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461256 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.461257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.462637 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.462638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.463241 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.463302 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463307 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:07:10.463363 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 23:07:10.463401 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":137,"thread_start_us":79,"threads_started":1}
I20260928 23:07:10.463536 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.15.212.65:35637)
0928 23:07:10.459940 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.459942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.459948 (+ 6us) server_negotiation.cc:407] Connection header received
0928 23:07:10.459987 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.459988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.460000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.460374 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:10.460413 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461169 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.462366 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.462910 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463393 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463408 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.463426 (+ 18us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.463467 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":55,"threads_started":1}
I20260928 23:07:10.463639 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.15.212.67:37281)
0928 23:07:10.460077 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.460079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.460185 (+ 106us) server_negotiation.cc:407] Connection header received
0928 23:07:10.460214 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.460215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.460227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.460298 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:10.460743 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461226 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.462630 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.463183 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463243 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463255 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.463517 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.463566 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":63,"threads_started":1}
I20260928 23:07:10.463716 23201 tablet_service.cc:1467] Processing CreateTablet for tablet ef7f87079650402c96e08ae685671b37 (DEFAULT_TABLE table=test-table [id=62b94e50370b424686c18b95ed8ec748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:10.463845 23201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7f87079650402c96e08ae685671b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:10.463841 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:37281 (local address 127.0.0.1:40704)
0928 23:07:10.460099 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.460107 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:10.460152 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:10.460352 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:10.460353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:10.460355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:10.460406 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.460407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461317 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.461318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.462532 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.462535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.463286 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.463350 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463355 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:07:10.463400 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 23:07:10.463442 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":78,"thread_start_us":65,"threads_started":1}
I20260928 23:07:10.464095 23474 tablet_service.cc:1467] Processing CreateTablet for tablet ef7f87079650402c96e08ae685671b37 (DEFAULT_TABLE table=test-table [id=62b94e50370b424686c18b95ed8ec748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:10.464195 23474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7f87079650402c96e08ae685671b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:10.465387 23642 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716: Bootstrap starting.
I20260928 23:07:10.465656 23643 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Bootstrap starting.
I20260928 23:07:10.465775 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.459788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.15.212.66:45219)
0928 23:07:10.460054 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.460057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.460071 (+ 14us) server_negotiation.cc:407] Connection header received
0928 23:07:10.460166 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.460167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.460178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.460241 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:10.460381 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.461162 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.462701 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.463148 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.463238 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.463249 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.463643 (+ 394us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.465675 (+ 2032us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":54,"threads_started":1}
I20260928 23:07:10.465970 23338 tablet_service.cc:1467] Processing CreateTablet for tablet ef7f87079650402c96e08ae685671b37 (DEFAULT_TABLE table=test-table [id=62b94e50370b424686c18b95ed8ec748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:10.466082 23338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef7f87079650402c96e08ae685671b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:10.466109 23642 tablet_bootstrap.cc:654] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:10.466109 23643 tablet_bootstrap.cc:654] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:10.466734 23643 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: No bootstrap required, opened a new log
I20260928 23:07:10.466801 23643 ts_tablet_manager.cc:1397] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:10.466920 23643 raft_consensus.cc:348] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.467000 23643 raft_consensus.cc:374] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:10.467022 23643 raft_consensus.cc:729] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38049d9a357149309c1e5891272a6c2b, State: Initialized, Role: FOLLOWER
I20260928 23:07:10.467074 23643 consensus_queue.cc:260] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.467236 23643 ts_tablet_manager.cc:1428] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:10.467613 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.467466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:40363)
0928 23:07:10.467559 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.467586 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260928 23:07:10.467638 23642 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716: No bootstrap required, opened a new log
I20260928 23:07:10.467680 23642 ts_tablet_manager.cc:1397] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:10.467850 23642 raft_consensus.cc:348] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.467895 23541 heartbeater.cc:503] Master 127.15.212.124:46229 was elected leader, sending a full tablet report...
I20260928 23:07:10.467963 23642 raft_consensus.cc:374] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:10.468017 23642 raft_consensus.cc:729] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd33f746945d48b3b9737ae7a52f4716, State: Initialized, Role: FOLLOWER
I20260928 23:07:10.468073 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.467940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:58945)
0928 23:07:10.468023 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.468047 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260928 23:07:10.468106 23642 consensus_queue.cc:260] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.468307 23642 ts_tablet_manager.cc:1428] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:10.468583 23647 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Bootstrap starting.
I20260928 23:07:10.468641 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.468533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:34467)
0928 23:07:10.468578 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.468614 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 23:07:10.468874 23268 heartbeater.cc:503] Master 127.15.212.124:46229 was elected leader, sending a full tablet report...
I20260928 23:07:10.469012 23647 tablet_bootstrap.cc:654] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:10.469172 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.469064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:33655)
0928 23:07:10.469116 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.469142 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 23:07:10.469794 23647 tablet_bootstrap.cc:492] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: No bootstrap required, opened a new log
I20260928 23:07:10.469837 23647 ts_tablet_manager.cc:1397] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:10.469960 23647 raft_consensus.cc:348] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.470027 23647 raft_consensus.cc:374] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:10.470041 23647 raft_consensus.cc:729] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0e2059b4fee46f296d35384db3e6a75, State: Initialized, Role: FOLLOWER
I20260928 23:07:10.470091 23647 consensus_queue.cc:260] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.470264 23647 ts_tablet_manager.cc:1428] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:10.470510 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.470415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:39177)
0928 23:07:10.470452 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.470481 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260928 23:07:10.470664 23405 heartbeater.cc:503] Master 127.15.212.124:46229 was elected leader, sending a full tablet report...
I20260928 23:07:10.470894 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.470808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:34675)
0928 23:07:10.470843 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.470864 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260928 23:07:10.617681 23650 raft_consensus.cc:482] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:10.617771 23650 raft_consensus.cc:504] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.618094 23650 leader_election.cc:288] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637), 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:10.622073 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.618339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:49581 (local address 127.15.212.67:37281)
0928 23:07:10.618383 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.618385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.618523 (+ 138us) server_negotiation.cc:407] Connection header received
0928 23:07:10.618577 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.618578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.618588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.618637 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:10.618800 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.619553 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621155 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.621708 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621785 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.621796 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.621957 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.621995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:07:10.622073 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.618327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:38073 (local address 127.15.212.65:35637)
0928 23:07:10.618375 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.618377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.618423 (+ 46us) server_negotiation.cc:407] Connection header received
0928 23:07:10.618521 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.618522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.618541 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.618603 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:10.618801 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.619556 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621170 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.621708 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621785 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.621796 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.621957 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.621995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 23:07:10.622109 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.618282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:37281 (local address 127.15.212.66:49581)
0928 23:07:10.618447 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.618456 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:10.618513 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:10.618655 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:10.618655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:10.618656 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:10.618724 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.618726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.619654 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.619654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.621059 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.621062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.621792 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.621873 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.621881 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 23:07:10.621938 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:07:10.621992 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":54,"thread_start_us":107,"threads_started":1}
I20260928 23:07:10.622104 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.618271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:35637 (local address 127.15.212.66:38073)
0928 23:07:10.618343 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.618354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:10.618433 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:10.618636 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:10.618637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:10.618640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:10.618724 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.618726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.619652 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.619653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.621059 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:10.621062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.621812 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.621812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:10.621873 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.621880 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 23:07:10.621938 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:07:10.621992 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":108}
I20260928 23:07:10.622486 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7f87079650402c96e08ae685671b37" candidate_uuid: "c0e2059b4fee46f296d35384db3e6a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd33f746945d48b3b9737ae7a52f4716" is_pre_election: true
I20260928 23:07:10.622504 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7f87079650402c96e08ae685671b37" candidate_uuid: "c0e2059b4fee46f296d35384db3e6a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38049d9a357149309c1e5891272a6c2b" is_pre_election: true
I20260928 23:07:10.622598 23221 raft_consensus.cc:2454] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e2059b4fee46f296d35384db3e6a75 in term 0.
I20260928 23:07:10.622618 23494 raft_consensus.cc:2454] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0e2059b4fee46f296d35384db3e6a75 in term 0.
I20260928 23:07:10.622773 23292 leader_election.cc:302] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0e2059b4fee46f296d35384db3e6a75, dd33f746945d48b3b9737ae7a52f4716; no voters:
I20260928 23:07:10.622898 23650 raft_consensus.cc:2781] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:10.622938 23650 raft_consensus.cc:482] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:10.622962 23650 raft_consensus.cc:3037] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:10.623643 23650 raft_consensus.cc:504] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.623790 23650 leader_election.cc:288] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [CANDIDATE]: Term 1 election: Requested vote from peers dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637), 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281)
I20260928 23:07:10.623919 23221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7f87079650402c96e08ae685671b37" candidate_uuid: "c0e2059b4fee46f296d35384db3e6a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd33f746945d48b3b9737ae7a52f4716"
I20260928 23:07:10.623945 23494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef7f87079650402c96e08ae685671b37" candidate_uuid: "c0e2059b4fee46f296d35384db3e6a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38049d9a357149309c1e5891272a6c2b"
I20260928 23:07:10.623989 23221 raft_consensus.cc:3037] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:10.624032 23494 raft_consensus.cc:3037] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:10.624624 23221 raft_consensus.cc:2454] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e2059b4fee46f296d35384db3e6a75 in term 1.
I20260928 23:07:10.624776 23292 leader_election.cc:302] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0e2059b4fee46f296d35384db3e6a75, dd33f746945d48b3b9737ae7a52f4716; no voters:
I20260928 23:07:10.624821 23494 raft_consensus.cc:2454] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0e2059b4fee46f296d35384db3e6a75 in term 1.
I20260928 23:07:10.624869 23650 raft_consensus.cc:2781] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:10.625026 23650 raft_consensus.cc:686] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 1 LEADER]: Becoming Leader. State: Replica: c0e2059b4fee46f296d35384db3e6a75, State: Running, Role: LEADER
I20260928 23:07:10.625123 23650 consensus_queue.cc:237] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } }
I20260928 23:07:10.625348 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.625251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:52595)
0928 23:07:10.625293 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.625319 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 23:07:10.625914 23088 catalog_manager.cc:5495] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66). New cstate: current_term: 1 leader_uuid: "c0e2059b4fee46f296d35384db3e6a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0e2059b4fee46f296d35384db3e6a75" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 } health_report { overall_health: UNKNOWN } } }
I20260928 23:07:10.643592 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.641234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.15.212.66:45219)
0928 23:07:10.641288 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:10.641290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:10.641300 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:10.641355 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:10.641356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:10.641374 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:10.641461 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:10.641580 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.642040 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.642838 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:10.643037 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:10.643115 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:10.643261 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:10.643369 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:10.643442 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:10.643499 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 23:07:10.643519 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 23:07:10.644583 23221 raft_consensus.cc:1261] T ef7f87079650402c96e08ae685671b37 P dd33f746945d48b3b9737ae7a52f4716 [term 1 FOLLOWER]: Refusing update from remote peer c0e2059b4fee46f296d35384db3e6a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:10.644647 23494 raft_consensus.cc:1261] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Refusing update from remote peer c0e2059b4fee46f296d35384db3e6a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:10.644826 23650 consensus_queue.cc:1035] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd33f746945d48b3b9737ae7a52f4716" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:10.644945 23652 consensus_queue.cc:1035] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38049d9a357149309c1e5891272a6c2b" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 37281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:10.645730 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.645623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:55019)
0928 23:07:10.645664 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.645698 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 23:07:10.645910 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.645774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:38545)
0928 23:07:10.645857 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.645883 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 23:07:10.646795 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.646677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:54723)
0928 23:07:10.646734 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.646761 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 23:07:10.647102 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.647000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.67:34637)
0928 23:07:10.647036 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.647069 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260928 23:07:10.647729 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.647639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.65:55237)
0928 23:07:10.647669 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.647693 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260928 23:07:10.647812 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.647237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:41451 (local address 127.15.212.66:35013)
0928 23:07:10.647764 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.647791 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:41451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512}
I20260928 23:07:10.648924 23088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:10.649013 23088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:10.650208 23088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3773c0660e474c498d93334f2113cc3f: Sending DeleteTablet for 3 replicas of tablet ef7f87079650402c96e08ae685671b37
I20260928 23:07:10.650434 23338 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7f87079650402c96e08ae685671b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:10 UTC) from {username='oryx', principal='oryx/127.15.212.124@KRBTEST.COM'} at 127.0.0.1:48866
I20260928 23:07:10.650473 23474 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7f87079650402c96e08ae685671b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:10 UTC) from {username='oryx', principal='oryx/127.15.212.124@KRBTEST.COM'} at 127.0.0.1:40704
I20260928 23:07:10.650631 23664 tablet_replica.cc:330] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: stopping tablet replica
I20260928 23:07:10.650656 23665 tablet_replica.cc:330] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: stopping tablet replica
I20260928 23:07:10.650714 23664 raft_consensus.cc:2229] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:10.650789 23665 raft_consensus.cc:2229] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:10.650816 23664 raft_consensus.cc:2258] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:10.650892 23665 raft_consensus.cc:2258] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:10.650920 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23050
I20260928 23:07:10.651684 23664 ts_tablet_manager.cc:1905] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:10.651692 23665 ts_tablet_manager.cc:1905] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:10.653623 23664 ts_tablet_manager.cc:1918] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:10.653630 23665 ts_tablet_manager.cc:1918] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:10.653676 23664 log.cc:1196] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/ef7f87079650402c96e08ae685671b37
I20260928 23:07:10.653679 23665 log.cc:1196] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/ef7f87079650402c96e08ae685671b37
I20260928 23:07:10.653925 23664 ts_tablet_manager.cc:1939] T ef7f87079650402c96e08ae685671b37 P c0e2059b4fee46f296d35384db3e6a75: Deleting consensus metadata
I20260928 23:07:10.653931 23665 ts_tablet_manager.cc:1939] T ef7f87079650402c96e08ae685671b37 P 38049d9a357149309c1e5891272a6c2b: Deleting consensus metadata
I20260928 23:07:10.654234 23074 catalog_manager.cc:4841] TS 38049d9a357149309c1e5891272a6c2b (127.15.212.67:37281): tablet ef7f87079650402c96e08ae685671b37 (table test-table [id=62b94e50370b424686c18b95ed8ec748]) successfully deleted
I20260928 23:07:10.654830 23074 catalog_manager.cc:4841] TS c0e2059b4fee46f296d35384db3e6a75 (127.15.212.66:45219): tablet ef7f87079650402c96e08ae685671b37 (table test-table [id=62b94e50370b424686c18b95ed8ec748]) successfully deleted
W20260928 23:07:10.657781 23073 connection.cc:535] client connection to 127.15.212.65:35637 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:07:10.658143 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:10.658046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:35637 (local address 127.0.0.1:53342)
0928 23:07:10.658085 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:10.658107 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:35637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260928 23:07:10.658211 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23275
W20260928 23:07:10.658341 23073 catalog_manager.cc:4584] TS dd33f746945d48b3b9737ae7a52f4716 (127.15.212.65:35637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef7f87079650402c96e08ae685671b37: Network error: Client connection negotiation failed: client connection to 127.15.212.65:35637: connect: Connection refused (error 111)
I20260928 23:07:10.663622 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23411
I20260928 23:07:10.668882 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22767
I20260928 23:07:10.673599 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 22957
2026-09-28T23:07:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7422 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:10 dist-test-slave-x2t8 krb5kdc[23668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:10 dist-test-slave-x2t8 krb5kdc[23668](info): set up 2 sockets
Sep 28 23:07:10 dist-test-slave-x2t8 krb5kdc[23668](info): commencing operation
krb5kdc: starting...
W20260928 23:07:11.708824 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:11 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab.
Entry for principal oryx/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab.
I20260928 23:07:11.819151 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44467
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab
--principal=oryx/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:44467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:11.897120 23681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:11.897310 23681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:11.897332 23681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:11.898475 23681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:11.898517 23681 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 23:07:11.898531 23681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:11.898545 23681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:11.898592 23681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:11.898623 23681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:11.900087 23681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:44467
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.126.keytab
--principal=oryx/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:11.900326 23681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:11.900570 23681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:11.905311 23689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:11.905337 23681 server_base.cc:1034] running on GCE node
W20260928 23:07:11.905288 23687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:11.905308 23686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:11.905773 23681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:11.906068 23681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:11.907240 23681 hybrid_clock.cc:648] HybridClock initialized: now 1790636831907225 us; error 26 us; skew 500 ppm
Sep 28 23:07:11 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636831, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:11.909754 23681 init.cc:377] Logged in from keytab as oryx/127.15.212.126@KRBTEST.COM (short username oryx)
I20260928 23:07:11.910210 23681 webserver.cc:460] Webserver started at http://127.15.212.126:44407/ using document root <none> and password file <none>
I20260928 23:07:11.910444 23681 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:11.910487 23681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:11.910638 23681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:11.911649 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "76204f84b5284b5e9647db1c4e747fc7"
format_stamp: "Formatted at 2026-09-28 23:07:11 on dist-test-slave-x2t8"
server_key: "14d6ea96bc2ea51029ca3bb73ca32718"
server_key_iv: "8772b6a11ce25778451a6c5edb78d916"
server_key_version: "encryptionkey@0"
I20260928 23:07:11.912009 23681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "76204f84b5284b5e9647db1c4e747fc7"
format_stamp: "Formatted at 2026-09-28 23:07:11 on dist-test-slave-x2t8"
server_key: "14d6ea96bc2ea51029ca3bb73ca32718"
server_key_iv: "8772b6a11ce25778451a6c5edb78d916"
server_key_version: "encryptionkey@0"
I20260928 23:07:11.913406 23681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:11.914077 23696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:11.914256 23681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:11.914319 23681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "76204f84b5284b5e9647db1c4e747fc7"
format_stamp: "Formatted at 2026-09-28 23:07:11 on dist-test-slave-x2t8"
server_key: "14d6ea96bc2ea51029ca3bb73ca32718"
server_key_iv: "8772b6a11ce25778451a6c5edb78d916"
server_key_version: "encryptionkey@0"
I20260928 23:07:11.914405 23681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:11.931474 23681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:11.933658 23681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:11.933820 23681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:11.938464 23681 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:44467
I20260928 23:07:11.938515 23758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:44467 every 8 connection(s)
I20260928 23:07:11.938802 23681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:11.939359 23759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:11.941562 23759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Bootstrap starting.
I20260928 23:07:11.942133 23759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:11.942430 23759 log.cc:824] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:11.943053 23759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: No bootstrap required, opened a new log
I20260928 23:07:11.943295 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23681
I20260928 23:07:11.943372 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:11.943574 16209 external_mini_cluster.cc:1428] Setting key 3efcc0bc96048f3a03e0119d16890d32
I20260928 23:07:11.945403 23759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } }
I20260928 23:07:11.945555 23759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:11.945621 23759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76204f84b5284b5e9647db1c4e747fc7, State: Initialized, Role: FOLLOWER
I20260928 23:07:11.945748 23759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } }
I20260928 23:07:11.945844 23759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:11.945888 23759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:11.945938 23759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:11.946447 23759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } }
I20260928 23:07:11.946570 23759 leader_election.cc:302] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76204f84b5284b5e9647db1c4e747fc7; no voters:
I20260928 23:07:11.946739 23759 leader_election.cc:288] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:11.946767 23764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:11.946888 23764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [term 1 LEADER]: Becoming Leader. State: Replica: 76204f84b5284b5e9647db1c4e747fc7, State: Running, Role: LEADER
I20260928 23:07:11.946983 23759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:11.946988 23764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } }
I20260928 23:07:11.947391 23764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76204f84b5284b5e9647db1c4e747fc7. Latest consensus state: current_term: 1 leader_uuid: "76204f84b5284b5e9647db1c4e747fc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } } }
I20260928 23:07:11.947399 23765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76204f84b5284b5e9647db1c4e747fc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76204f84b5284b5e9647db1c4e747fc7" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44467 } } }
I20260928 23:07:11.948520 23765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:11.948731 23764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7 [sys.catalog]: This master's current role is: LEADER
W20260928 23:07:11.948845 23779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:07:11.948882 23779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:07:11.948966 23781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:11.949117 23781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:11.950644 23781 catalog_manager.cc:1347] Generated new cluster ID: 7268929067074b9e9dab045b6844101b
I20260928 23:07:11.950690 23781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:07:11 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:11.955515 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:11.944052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.15.212.126:44467)
0928 23:07:11.944228 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:11.944238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:11.944270 (+ 32us) server_negotiation.cc:407] Connection header received
0928 23:07:11.945394 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:11.945401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:11.945445 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:11.945581 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:11.946275 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:11.947652 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:11.948755 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:11.949361 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:11.949463 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:11.951970 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:11.951974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:11.951974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:11.951985 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:11.954052 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:11.954424 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:11.954427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:11.954427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:11.954532 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:11.954741 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:11.954742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:11.954742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:11.954957 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:11.955132 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:11.955197 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 23:07:11.955330 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260928 23:07:11.970340 23781 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:11.970873 23781 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:11.974783 23781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Generated new TSK 0
I20260928 23:07:11.974977 23781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
I20260928 23:07:12.003249 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44467
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:12.082022 23785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:12.082249 23785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:12.082286 23785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:12.083420 23785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:12.083484 23785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:12.083525 23785 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:07:12.083562 23785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:12.085124 23785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:12.085395 23785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:12.085635 23785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:12.090757 23793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.090754 23790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.090837 23791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:12.090997 23785 server_base.cc:1034] running on GCE node
I20260928 23:07:12.091223 23785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:12.091480 23785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:12.092715 23785 hybrid_clock.cc:648] HybridClock initialized: now 1790636832092673 us; error 32 us; skew 500 ppm
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:12.095170 23785 init.cc:377] Logged in from keytab as oryx/127.15.212.65@KRBTEST.COM (short username oryx)
I20260928 23:07:12.095629 23785 webserver.cc:460] Webserver started at http://127.15.212.65:38913/ using document root <none> and password file <none>
I20260928 23:07:12.095857 23785 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:12.095901 23785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:12.096074 23785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:12.097123 23785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "a31acb3121bdcc5c65f1acd8523cf008"
server_key_iv: "fc39f625c641936bbf39d61688c4d6ed"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.097505 23785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "a31acb3121bdcc5c65f1acd8523cf008"
server_key_iv: "fc39f625c641936bbf39d61688c4d6ed"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.098803 23785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:12.099496 23800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.099676 23785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:12.099745 23785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "a31acb3121bdcc5c65f1acd8523cf008"
server_key_iv: "fc39f625c641936bbf39d61688c4d6ed"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.099835 23785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:12.106702 23785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:12.108394 23785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:12.108603 23785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:12.109046 23785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:12.109086 23785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.109107 23785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:12.109143 23785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.115401 23785 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:43417
I20260928 23:07:12.115458 23915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:43417 every 8 connection(s)
I20260928 23:07:12.115782 23785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.117664 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23785
I20260928 23:07:12.117795 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:12.118039 16209 external_mini_cluster.cc:1428] Setting key 8930e11b0b97e6764fdb86f27816da22
I20260928 23:07:12.119192 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.109466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.65:43183)
0928 23:07:12.109638 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.109675 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.110125 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.110409 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.110413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.110765 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.111038 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.111045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.112073 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.112074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.113388 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.113393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.113923 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.113924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.114059 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.114557 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.114566 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.116261 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.117742 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.117750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.117752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.118077 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.118338 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.118339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.118339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.118445 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.118732 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.118738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.118852 (+ 114us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.118936 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.119050 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":114,"threads_started":1}
I20260928 23:07:12.119227 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.109547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:43183 (local address 127.15.212.126:44467)
0928 23:07:12.109618 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.109621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.109755 (+ 134us) server_negotiation.cc:407] Connection header received
0928 23:07:12.110215 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.110217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.110234 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.110308 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.111139 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.111959 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.113498 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.113832 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.113914 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.116349 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.116351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.116351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.116371 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.117666 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.118175 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.118176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.118176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.118239 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.118531 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.118532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.118532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.118648 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.118732 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.118982 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.119087 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:07:12.119669 23916 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44467
I20260928 23:07:12.119761 23916 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:12.119933 23916 heartbeater.cc:511] Master 127.15.212.126:44467 requested a full tablet report, sending...
I20260928 23:07:12.120522 23712 ts_manager.cc:194] Registered new tserver with Master: 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417)
I20260928 23:07:12.121325 23712 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:43183
WARNING: no policy specified for oryx/127.15.212.66@KRBTEST.COM; defaulting to no policy
I20260928 23:07:12.128041 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.121156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.65:45607)
0928 23:07:12.121290 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.121302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.121370 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.121534 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.121534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.121765 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.121832 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.121834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.122566 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.122567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.123480 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.123483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.124004 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.124005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.124093 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.124613 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.124618 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.124977 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.126931 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.126932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.126933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.127144 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.127386 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.127387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.127387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.127457 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.127719 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.127719 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.127779 (+ 60us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.127842 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.127942 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260928 23:07:12.128217 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.121202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:45607 (local address 127.15.212.126:44467)
0928 23:07:12.121239 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.121240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.121349 (+ 109us) server_negotiation.cc:407] Connection header received
0928 23:07:12.121433 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.121434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.121444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.121479 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.121913 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.122442 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.123577 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.123916 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.123994 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.125089 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.125090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.125091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.125097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.126819 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.127235 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.127236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.127236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.127319 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.127544 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.127545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.127546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.127664 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.127726 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.127865 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.127942 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
I20260928 23:07:12.144950 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44467
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:12.224215 23925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:12.224429 23925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:12.224480 23925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:12.226418 23925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:12.226461 23925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:12.226482 23925 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:07:12.226500 23925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:12.227941 23925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:12.228120 23925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:12.228283 23925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:12.233104 23933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.233100 23931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.233150 23930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:12.233299 23925 server_base.cc:1034] running on GCE node
I20260928 23:07:12.233518 23925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:12.233779 23925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:12.234947 23925 hybrid_clock.cc:648] HybridClock initialized: now 1790636832234935 us; error 28 us; skew 500 ppm
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:12.237453 23925 init.cc:377] Logged in from keytab as oryx/127.15.212.66@KRBTEST.COM (short username oryx)
I20260928 23:07:12.237927 23925 webserver.cc:460] Webserver started at http://127.15.212.66:38341/ using document root <none> and password file <none>
I20260928 23:07:12.238116 23925 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:12.238157 23925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:12.238238 23925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:12.239162 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "71ded63256b44da292b61809e0eca1bf"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "f63d4b1c1fd080fc3bbfa1effdaae309"
server_key_iv: "1b2cd0edaadcf3c45a20bf3dc4399453"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.239435 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "71ded63256b44da292b61809e0eca1bf"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "f63d4b1c1fd080fc3bbfa1effdaae309"
server_key_iv: "1b2cd0edaadcf3c45a20bf3dc4399453"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.240675 23925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:07:12.241302 23940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.241473 23925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:12.241554 23925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "71ded63256b44da292b61809e0eca1bf"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "f63d4b1c1fd080fc3bbfa1effdaae309"
server_key_iv: "1b2cd0edaadcf3c45a20bf3dc4399453"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.241609 23925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:12.265216 23925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:12.266984 23925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:12.267093 23925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:12.267496 23925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:12.267529 23925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.267556 23925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:12.267570 23925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.274525 23925 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:41501
I20260928 23:07:12.274693 24055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:41501 every 8 connection(s)
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.274937 23925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:12.279026 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.268039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:42097 (local address 127.15.212.126:44467)
0928 23:07:12.268102 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.268108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.268317 (+ 209us) server_negotiation.cc:407] Connection header received
0928 23:07:12.268905 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.268906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.268920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.268969 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.269922 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.270752 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.272421 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.272764 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.272822 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.275483 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.275485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.275485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.275492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.277364 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.277897 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.277898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.277899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.277963 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.278248 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.278249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.278250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.278418 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.278499 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.278860 (+ 361us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.278939 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 23:07:12.279192 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.268027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.66:42097)
0928 23:07:12.268200 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.268235 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.268784 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.269074 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.269080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.269496 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.269794 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.269799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.270851 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.270852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.272319 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.272321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.272846 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.272847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.273015 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.273567 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.273578 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.275399 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.277452 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.277459 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.277461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.277798 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.278044 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.278045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.278045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.278172 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.278505 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.278512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.278705 (+ 193us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.278822 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.278971 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260928 23:07:12.279191 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 23925
I20260928 23:07:12.279291 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:12.279508 16209 external_mini_cluster.cc:1428] Setting key dc17613635faaad611958bc5d780c923
I20260928 23:07:12.279685 24056 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44467
I20260928 23:07:12.279753 24056 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:12.279922 24056 heartbeater.cc:511] Master 127.15.212.126:44467 requested a full tablet report, sending...
I20260928 23:07:12.280674 23713 ts_manager.cc:194] Registered new tserver with Master: 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:12.281116 23713 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:42097
I20260928 23:07:12.286491 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.280679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.66:41293)
0928 23:07:12.280867 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.280877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.280936 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.281055 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.281056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.281332 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.281401 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.281403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.282188 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.282189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.282883 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.282885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.283201 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.283203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.283272 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.283696 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.283700 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.283948 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.285455 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.285456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.285456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.285626 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.285835 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.285836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.285836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.285892 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.286172 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.286173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.286216 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.286304 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.286399 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260928 23:07:12.286486 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.280664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:41293 (local address 127.15.212.126:44467)
0928 23:07:12.280698 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.280700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.280946 (+ 246us) server_negotiation.cc:407] Connection header received
0928 23:07:12.280985 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.280986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.280995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.281051 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.281481 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.282085 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.282954 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.283127 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.283202 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.284044 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.284045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.284045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.284052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.285360 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.285711 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.285712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.285713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.285763 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.285962 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.285963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.285963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.286101 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.286180 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.286326 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.286397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for oryx/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
I20260928 23:07:12.307744 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44467
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:12.387692 24065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:12.387879 24065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:12.387902 24065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:12.389084 24065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:12.389125 24065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:12.389146 24065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 23:07:12.389200 24065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:12.390648 24065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.212.126:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:12.390874 24065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:12.391100 24065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:12.395980 24073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.395980 24070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:12.396107 24071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:12.396225 24065 server_base.cc:1034] running on GCE node
I20260928 23:07:12.396487 24065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:12.396798 24065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:12.397953 24065 hybrid_clock.cc:648] HybridClock initialized: now 1790636832397933 us; error 36 us; skew 500 ppm
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:12.400733 24065 init.cc:377] Logged in from keytab as oryx/127.15.212.67@KRBTEST.COM (short username oryx)
I20260928 23:07:12.401175 24065 webserver.cc:460] Webserver started at http://127.15.212.67:36097/ using document root <none> and password file <none>
I20260928 23:07:12.401414 24065 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:12.401463 24065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:12.401595 24065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:12.402674 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "6c11e539122b410285b32fa6f9d31b81"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "fb8e5a5cc42135099576b8c0c748da78"
server_key_iv: "c2b9fed967e882f0d20e1150efef67b3"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.403047 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "6c11e539122b410285b32fa6f9d31b81"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "fb8e5a5cc42135099576b8c0c748da78"
server_key_iv: "c2b9fed967e882f0d20e1150efef67b3"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.404347 24065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:12.405187 24080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.405371 24065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:12.405443 24065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "6c11e539122b410285b32fa6f9d31b81"
format_stamp: "Formatted at 2026-09-28 23:07:12 on dist-test-slave-x2t8"
server_key: "fb8e5a5cc42135099576b8c0c748da78"
server_key_iv: "c2b9fed967e882f0d20e1150efef67b3"
server_key_version: "encryptionkey@0"
I20260928 23:07:12.405555 24065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:12.426427 24065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:12.428313 24065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:12.428490 24065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:12.429035 24065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:12.429076 24065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.429100 24065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:12.429148 24065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.436480 24065 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:40843
I20260928 23:07:12.436609 24195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:40843 every 8 connection(s)
I20260928 23:07:12.436883 24065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.440059 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.429532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:49485 (local address 127.15.212.126:44467)
0928 23:07:12.429595 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.429598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.429902 (+ 304us) server_negotiation.cc:407] Connection header received
0928 23:07:12.431047 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.431049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.431064 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.431132 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.432100 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.432824 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.434482 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.434732 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.434804 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.437491 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.437493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.437493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.437499 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.438692 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.439112 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.439113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.439114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.439173 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.439438 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.439439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.439439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.439559 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.439628 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.439892 (+ 264us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.439972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:12.440248 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.429552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.67:49485)
0928 23:07:12.429744 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.429795 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.430780 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.431268 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.431274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.431683 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.431982 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.431989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.432911 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.432912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.434382 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.434385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.434827 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.434828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.434983 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.435537 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.435547 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.437388 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.438798 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.438804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.438806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.439024 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.439249 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.439250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.439250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.439366 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.439630 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.439636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.439765 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.439869 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.440014 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260928 23:07:12.440742 24196 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44467
I20260928 23:07:12.440822 24196 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:12.441020 24196 heartbeater.cc:511] Master 127.15.212.126:44467 requested a full tablet report, sending...
I20260928 23:07:12.441428 23713 ts_manager.cc:194] Registered new tserver with Master: 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843)
I20260928 23:07:12.441812 23713 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:49485
I20260928 23:07:12.442356 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24065
I20260928 23:07:12.442472 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:12.442694 16209 external_mini_cluster.cc:1428] Setting key d1a47076ee0b1f23bf5c92eaed62f052
I20260928 23:07:12.443572 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:07:12.448365 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.442088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48773 (local address 127.15.212.126:44467)
0928 23:07:12.442126 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.442128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.442279 (+ 151us) server_negotiation.cc:407] Connection header received
0928 23:07:12.442338 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.442339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.442349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.442408 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.442794 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.443424 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.444488 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.444692 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.444748 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.445748 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.445749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.445749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.445756 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.447294 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.447666 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.447667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.447668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.447726 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.447884 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.447885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.447885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.448016 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.448085 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.448197 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.448269 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:07:12.448611 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.442038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.15.212.67:48773)
0928 23:07:12.442202 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.442215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.442275 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.442411 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.442412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.442635 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.442710 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.442712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.443517 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.443518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.444385 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.444387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.444766 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.444767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.444838 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.445268 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.445273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.445658 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.447380 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.447381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.447382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.447587 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.447787 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.447788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.447788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.447833 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.448082 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.448083 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.448117 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.448169 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.448295 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 23:07:12.465145 23763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:12.460047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.15.212.126:44467)
0928 23:07:12.460089 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.460091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.460133 (+ 42us) server_negotiation.cc:407] Connection header received
0928 23:07:12.460188 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.460189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.460197 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.460243 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.460818 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.461283 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.461964 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.462137 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.462184 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.465053 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53340: BlockingRecv error: recv got EOF from 127.0.0.1:53340 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.471971 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.465912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.15.212.126:44467)
0928 23:07:12.465993 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.465995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.466003 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:12.466045 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.466046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.466054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.466094 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.466494 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.466927 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.467547 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.467718 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.467773 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.469690 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.469691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.469692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.469698 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.471076 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.471359 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.471360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.471360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.471406 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.471548 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.471549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.471549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.471631 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.471700 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.471861 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.471912 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:07:12.473544 23713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:12.475066 23713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:12.481312 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43417 (local address 127.0.0.1:43664)
0928 23:07:12.477429 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.477486 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.477552 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.477848 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.477850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.477852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.477919 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.477920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.478930 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.478931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.480037 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.480039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.480986 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.480987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.481074 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481089 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.481169 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.481219 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":44,"threads_started":1}
I20260928 23:07:12.481323 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.15.212.65:43417)
0928 23:07:12.477663 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.477668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.477679 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:07:12.477717 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.477719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.477736 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.477823 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.478030 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.478843 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.480132 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.480905 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.480976 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481048 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.481191 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.481229 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260928 23:07:12.481726 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41501 (local address 127.0.0.1:36234)
0928 23:07:12.477616 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.477627 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.477683 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.478176 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.478177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.478179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.478208 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.478209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.479417 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.479418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.480423 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.480425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.481493 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.481494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.481557 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481560 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.481604 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.481640 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":36,"thread_start_us":84,"threads_started":1}
I20260928 23:07:12.482043 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.15.212.66:41501)
0928 23:07:12.477965 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.477969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.477979 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:12.478015 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.478017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.478033 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.478109 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.478284 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.479330 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.480534 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.481244 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.481304 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481870 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.481900 (+ 30us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.481940 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":108,"threads_started":1}
I20260928 23:07:12.482123 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.15.212.67:40843)
0928 23:07:12.477720 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.477724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.477828 (+ 104us) server_negotiation.cc:407] Connection header received
0928 23:07:12.477869 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.477871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.477887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.477974 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.478188 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.479313 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.480748 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.481614 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.481683 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481748 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.481993 (+ 245us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.482036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260928 23:07:12.481997 23850 tablet_service.cc:1467] Processing CreateTablet for tablet b2f274cf44cb439e806933d5a9fe7828 (DEFAULT_TABLE table=test-table [id=ea3340f35864478ba23b2511cf787bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.481915 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.477294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.0.0.1:53590)
0928 23:07:12.477756 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.477767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.477814 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.478050 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.478051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.478054 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.478107 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.478109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.479419 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.479420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.480639 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.480642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.481694 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.481694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.481750 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.481753 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.481812 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.481852 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":93,"thread_start_us":82,"threads_started":1}
I20260928 23:07:12.482379 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f274cf44cb439e806933d5a9fe7828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.482717 23990 tablet_service.cc:1467] Processing CreateTablet for tablet b2f274cf44cb439e806933d5a9fe7828 (DEFAULT_TABLE table=test-table [id=ea3340f35864478ba23b2511cf787bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.483018 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f274cf44cb439e806933d5a9fe7828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.485791 24224 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Bootstrap starting.
I20260928 23:07:12.486059 24225 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Bootstrap starting.
I20260928 23:07:12.486045 24130 tablet_service.cc:1467] Processing CreateTablet for tablet b2f274cf44cb439e806933d5a9fe7828 (DEFAULT_TABLE table=test-table [id=ea3340f35864478ba23b2511cf787bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.486361 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2f274cf44cb439e806933d5a9fe7828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.486438 24224 tablet_bootstrap.cc:654] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.486698 24225 tablet_bootstrap.cc:654] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.486752 24224 log.cc:824] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:12.487049 24225 log.cc:824] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:12.487464 24224 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: No bootstrap required, opened a new log
I20260928 23:07:12.487519 24224 ts_tablet_manager.cc:1397] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:12.489085 24224 raft_consensus.cc:348] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.489208 24224 raft_consensus.cc:374] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.489229 24224 raft_consensus.cc:729] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.489284 24225 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: No bootstrap required, opened a new log
I20260928 23:07:12.489393 24225 ts_tablet_manager.cc:1397] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 23:07:12.489388 24224 consensus_queue.cc:260] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.489637 24224 ts_tablet_manager.cc:1428] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:07:12.489789 23916 heartbeater.cc:503] Master 127.15.212.126:44467 was elected leader, sending a full tablet report...
I20260928 23:07:12.491144 24225 raft_consensus.cc:348] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.491259 24225 raft_consensus.cc:374] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.491279 24225 raft_consensus.cc:729] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.491384 24225 consensus_queue.cc:260] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.491587 24225 ts_tablet_manager.cc:1428] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:07:12.491655 24056 heartbeater.cc:503] Master 127.15.212.126:44467 was elected leader, sending a full tablet report...
I20260928 23:07:12.491778 24228 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Bootstrap starting.
I20260928 23:07:12.492362 24228 tablet_bootstrap.cc:654] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.492755 24228 log.cc:824] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:12.493400 24228 tablet_bootstrap.cc:492] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: No bootstrap required, opened a new log
I20260928 23:07:12.493451 24228 ts_tablet_manager.cc:1397] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:12.494688 24228 raft_consensus.cc:348] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.494803 24228 raft_consensus.cc:374] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.494823 24228 raft_consensus.cc:729] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.494927 24228 consensus_queue.cc:260] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.495189 24228 ts_tablet_manager.cc:1428] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:07:12.495218 24196 heartbeater.cc:503] Master 127.15.212.126:44467 was elected leader, sending a full tablet report...
W20260928 23:07:12.525496 24057 tablet.cc:2369] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:12.609005 24229 raft_consensus.cc:482] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:12.609153 24229 raft_consensus.cc:504] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.609565 24229 leader_election.cc:288] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501), 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843)
I20260928 23:07:12.610411 24232 raft_consensus.cc:482] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:12.610523 24232 raft_consensus.cc:504] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.610837 24232 leader_election.cc:288] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:12.613119 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.609740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41501 (local address 127.15.212.65:50995)
0928 23:07:12.609792 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.609805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.609855 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.610038 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.610038 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.610040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.610113 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.610115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.610842 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.610843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.612088 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.612090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612848 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.612932 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.612936 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.612984 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.613031 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 23:07:12.613252 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.609843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:34359 (local address 127.15.212.67:40843)
0928 23:07:12.609898 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.609900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.610024 (+ 124us) server_negotiation.cc:407] Connection header received
0928 23:07:12.610092 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.610093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.610102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.610190 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.610392 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.611045 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612344 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612888 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612933 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.612940 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.613156 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.613190 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:07:12.613492 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.609740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.15.212.65:34359)
0928 23:07:12.609956 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.609964 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.610038 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.610196 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.610197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.610199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.610272 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.610274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.611164 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.611165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.612258 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.612260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612995 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.613065 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.613068 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.613357 (+ 289us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.613396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":71,"thread_start_us":79,"threads_started":1}
I20260928 23:07:12.613516 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.609845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:50995 (local address 127.15.212.66:41501)
0928 23:07:12.609909 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.609911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.609920 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:12.609954 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.609955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.609963 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.610035 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.610185 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.610764 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612169 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612717 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612783 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.612793 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.613415 (+ 622us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.613453 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:07:12.613931 24010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf" is_pre_election: true
I20260928 23:07:12.613924 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c11e539122b410285b32fa6f9d31b81" is_pre_election: true
I20260928 23:07:12.614032 24150 raft_consensus.cc:2454] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in term 0.
I20260928 23:07:12.614058 24010 raft_consensus.cc:2454] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in term 0.
I20260928 23:07:12.614248 23803 leader_election.cc:302] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c918286f0504d71ba0ceca9c88f1ff9, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.614383 24229 raft_consensus.cc:2781] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:12.614435 24229 raft_consensus.cc:482] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:12.614462 24229 raft_consensus.cc:3037] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.615100 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.611079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41501 (local address 127.15.212.67:46999)
0928 23:07:12.611655 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.611664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.611720 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.611882 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.611883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.611884 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.611945 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.611946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612840 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.614325 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.614328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.614911 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.614912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.614966 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.614968 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.615010 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.615035 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260928 23:07:12.615186 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.611223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:46999 (local address 127.15.212.66:41501)
0928 23:07:12.611400 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.611402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.611701 (+ 299us) server_negotiation.cc:407] Connection header received
0928 23:07:12.611783 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.611784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.611792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.611831 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.612008 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612742 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.614406 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.614825 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.614862 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.614866 (+ 4us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.615049 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.615097 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260928 23:07:12.615360 24010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf" is_pre_election: true
I20260928 23:07:12.615433 24010 raft_consensus.cc:2454] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:12.615612 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.610964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43417 (local address 127.15.212.67:47149)
0928 23:07:12.611450 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.611460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.611543 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.611698 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.611699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.611701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.611750 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.611751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612558 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.612559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.613670 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.613673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.615367 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.615368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.615445 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.615448 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.615495 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.615547 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":17}
I20260928 23:07:12.615581 24229 raft_consensus.cc:504] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.615715 24229 leader_election.cc:288] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501), 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843)
I20260928 23:07:12.615820 24083 leader_election.cc:302] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.615914 24010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf"
I20260928 23:07:12.615926 24232 raft_consensus.cc:2781] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:12.615973 24232 raft_consensus.cc:482] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:12.616005 24232 raft_consensus.cc:3037] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.615980 24010 raft_consensus.cc:3037] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.615993 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c11e539122b410285b32fa6f9d31b81"
I20260928 23:07:12.616099 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.611334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:47149 (local address 127.15.212.65:43417)
0928 23:07:12.611396 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.611398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.611515 (+ 117us) server_negotiation.cc:407] Connection header received
0928 23:07:12.611591 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.611592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.611600 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.611661 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.611812 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.612486 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.613736 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.614241 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.614306 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.614313 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.615989 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.616029 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:07:12.616349 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" is_pre_election: true
I20260928 23:07:12.616490 23870 raft_consensus.cc:2379] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c11e539122b410285b32fa6f9d31b81 in current term 1: Already voted for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in this term.
W20260928 23:07:12.616555 23917 tablet.cc:2369] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:12.616894 24232 raft_consensus.cc:504] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.617018 24150 raft_consensus.cc:2379] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0c918286f0504d71ba0ceca9c88f1ff9 in current term 1: Already voted for candidate 6c11e539122b410285b32fa6f9d31b81 in this term.
I20260928 23:07:12.617165 24010 raft_consensus.cc:2454] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in term 1.
I20260928 23:07:12.617164 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
I20260928 23:07:12.617241 23870 raft_consensus.cc:2379] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c11e539122b410285b32fa6f9d31b81 in current term 1: Already voted for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in this term.
I20260928 23:07:12.617158 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2f274cf44cb439e806933d5a9fe7828" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf"
I20260928 23:07:12.617360 24009 raft_consensus.cc:2379] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c11e539122b410285b32fa6f9d31b81 in current term 1: Already voted for candidate 0c918286f0504d71ba0ceca9c88f1ff9 in this term.
I20260928 23:07:12.617033 24232 leader_election.cc:288] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Requested vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:12.617386 23803 leader_election.cc:302] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c918286f0504d71ba0ceca9c88f1ff9, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.617453 24229 raft_consensus.cc:2781] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:12.617554 24083 leader_election.cc:302] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81; no voters: 0c918286f0504d71ba0ceca9c88f1ff9, 71ded63256b44da292b61809e0eca1bf
I20260928 23:07:12.617604 24229 raft_consensus.cc:686] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Running, Role: LEADER
I20260928 23:07:12.617674 24229 consensus_queue.cc:237] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.617878 24232 raft_consensus.cc:2726] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260928 23:07:12.618485 23713 catalog_manager.cc:5495] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65). New cstate: current_term: 1 leader_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260928 23:07:12.624038 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.622020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.15.212.65:43417)
0928 23:07:12.622071 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.622073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.622080 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:12.622146 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.622147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.622160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.622218 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:12.622316 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.622751 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.623423 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.623589 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.623637 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.623775 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.623859 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.623913 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.623946 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.623960 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:12.625613 24150 raft_consensus.cc:1261] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Refusing update from remote peer 0c918286f0504d71ba0ceca9c88f1ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.625648 24009 raft_consensus.cc:1261] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Refusing update from remote peer 0c918286f0504d71ba0ceca9c88f1ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.625895 24229 consensus_queue.cc:1035] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.626004 24236 consensus_queue.cc:1035] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.628824 24240 mvcc.cc:205] Tried to move back new op lower bound from 7334448466432294912 to 7334448466402672640. Current Snapshot: MvccSnapshot[applied={T|T < 7334448466432294912 or (T in {7334448466432294912})}]
I20260928 23:07:12.628930 24241 mvcc.cc:205] Tried to move back new op lower bound from 7334448466432294912 to 7334448466402672640. Current Snapshot: MvccSnapshot[applied={T|T < 7334448466432294912}]
I20260928 23:07:12.630667 23712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:12.630759 23712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:12.631474 23712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Sending DeleteTablet for 3 replicas of tablet b2f274cf44cb439e806933d5a9fe7828
I20260928 23:07:12.631769 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f274cf44cb439e806933d5a9fe7828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:43664
I20260928 23:07:12.631820 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f274cf44cb439e806933d5a9fe7828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53590
I20260928 23:07:12.631940 24249 tablet_replica.cc:330] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: stopping tablet replica
I20260928 23:07:12.632025 24249 raft_consensus.cc:2229] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:12.632143 24249 raft_consensus.cc:2258] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.632117 23990 tablet_service.cc:1514] Processing DeleteTablet for tablet b2f274cf44cb439e806933d5a9fe7828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36234
I20260928 23:07:12.632335 24251 tablet_replica.cc:330] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: stopping tablet replica
I20260928 23:07:12.632483 24251 raft_consensus.cc:2229] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:12.632396 23712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:12.632556 23712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:12.632663 24249 ts_tablet_manager.cc:1905] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.632834 24251 raft_consensus.cc:2258] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.632853 24250 tablet_replica.cc:330] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: stopping tablet replica
I20260928 23:07:12.632956 24250 raft_consensus.cc:2229] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:12.633075 24250 raft_consensus.cc:2258] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.633790 23850 tablet_service.cc:1467] Processing CreateTablet for tablet 5f081a667c1641708c1e7b9a308d4fff (DEFAULT_TABLE table=test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.633798 23990 tablet_service.cc:1467] Processing CreateTablet for tablet 5f081a667c1641708c1e7b9a308d4fff (DEFAULT_TABLE table=test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.633898 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f081a667c1641708c1e7b9a308d4fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.633898 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f081a667c1641708c1e7b9a308d4fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.635149 24225 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Bootstrap starting.
I20260928 23:07:12.635227 24224 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Bootstrap starting.
I20260928 23:07:12.635526 24225 tablet_bootstrap.cc:654] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.635653 24224 tablet_bootstrap.cc:654] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.635857 24251 ts_tablet_manager.cc:1905] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.636161 24225 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: No bootstrap required, opened a new log
I20260928 23:07:12.636204 24225 ts_tablet_manager.cc:1397] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:12.636200 24130 tablet_service.cc:1467] Processing CreateTablet for tablet 5f081a667c1641708c1e7b9a308d4fff (DEFAULT_TABLE table=test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.636307 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f081a667c1641708c1e7b9a308d4fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.636327 24225 raft_consensus.cc:348] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.636396 24225 raft_consensus.cc:374] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.636425 24225 raft_consensus.cc:729] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.636499 24225 consensus_queue.cc:260] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.636590 24225 ts_tablet_manager.cc:1428] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:12.637326 24228 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Bootstrap starting.
I20260928 23:07:12.637550 24250 ts_tablet_manager.cc:1905] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.637740 24228 tablet_bootstrap.cc:654] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.639015 24249 ts_tablet_manager.cc:1918] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:12.639038 24250 ts_tablet_manager.cc:1918] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:12.639096 24250 log.cc:1196] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/b2f274cf44cb439e806933d5a9fe7828
I20260928 23:07:12.639091 24249 log.cc:1196] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/b2f274cf44cb439e806933d5a9fe7828
I20260928 23:07:12.639031 24251 ts_tablet_manager.cc:1918] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:12.639267 24251 log.cc:1196] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/b2f274cf44cb439e806933d5a9fe7828
I20260928 23:07:12.639355 24249 ts_tablet_manager.cc:1939] T b2f274cf44cb439e806933d5a9fe7828 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting consensus metadata
I20260928 23:07:12.639364 24250 ts_tablet_manager.cc:1939] T b2f274cf44cb439e806933d5a9fe7828 P 6c11e539122b410285b32fa6f9d31b81: Deleting consensus metadata
I20260928 23:07:12.639618 24251 ts_tablet_manager.cc:1939] T b2f274cf44cb439e806933d5a9fe7828 P 71ded63256b44da292b61809e0eca1bf: Deleting consensus metadata
I20260928 23:07:12.639748 23697 catalog_manager.cc:4841] TS 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843): tablet b2f274cf44cb439e806933d5a9fe7828 (table test-table [id=ea3340f35864478ba23b2511cf787bda]) successfully deleted
I20260928 23:07:12.639745 23698 catalog_manager.cc:4841] TS 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417): tablet b2f274cf44cb439e806933d5a9fe7828 (table test-table [id=ea3340f35864478ba23b2511cf787bda]) successfully deleted
I20260928 23:07:12.640050 24224 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: No bootstrap required, opened a new log
I20260928 23:07:12.640100 24224 ts_tablet_manager.cc:1397] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 23:07:12.640193 24228 tablet_bootstrap.cc:492] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: No bootstrap required, opened a new log
I20260928 23:07:12.640239 24228 ts_tablet_manager.cc:1397] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 23:07:12.640234 24224 raft_consensus.cc:348] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.640323 24224 raft_consensus.cc:374] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.640352 24224 raft_consensus.cc:729] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.640388 24224 consensus_queue.cc:260] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.640368 24228 raft_consensus.cc:348] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.640426 24228 raft_consensus.cc:374] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.640477 24228 raft_consensus.cc:729] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.640484 24224 ts_tablet_manager.cc:1428] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.640515 24228 consensus_queue.cc:260] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.640596 24228 ts_tablet_manager.cc:1428] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:12.640782 23699 catalog_manager.cc:4841] TS 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501): tablet b2f274cf44cb439e806933d5a9fe7828 (table test-table [id=ea3340f35864478ba23b2511cf787bda]) successfully deleted
I20260928 23:07:12.667466 24230 raft_consensus.cc:482] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:12.667577 24230 raft_consensus.cc:504] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.667930 24230 leader_election.cc:288] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843)
I20260928 23:07:12.671504 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.668155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:43741 (local address 127.15.212.65:43417)
0928 23:07:12.668291 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.668292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.668302 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:12.668328 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.668329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.668350 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.668403 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.668618 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.669137 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.670557 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.671117 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.671188 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.671196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.671403 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.671443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260928 23:07:12.671650 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.668316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:56693 (local address 127.15.212.67:40843)
0928 23:07:12.668367 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.668369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.668376 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:12.668400 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.668401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.668409 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.668513 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:12.668716 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.669452 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.670657 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.671225 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.671294 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.671303 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.671545 (+ 242us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.671588 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:07:12.671512 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.668089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43417 (local address 127.15.212.66:43741)
0928 23:07:12.668213 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.668224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.668276 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.668418 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.668419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.668420 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.668541 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.668543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.669226 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.669227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.670464 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.670466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.671198 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.671200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.671278 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.671284 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.671387 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.671427 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260928 23:07:12.671825 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.668079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.15.212.66:56693)
0928 23:07:12.668179 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.668191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.668228 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.668557 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.668558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.668559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:12.668637 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.668638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.669565 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.669566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.670571 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.670573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.671308 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.671310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.671398 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.671401 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.671455 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.671494 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260928 23:07:12.672009 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f081a667c1641708c1e7b9a308d4fff" candidate_uuid: "71ded63256b44da292b61809e0eca1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c11e539122b410285b32fa6f9d31b81" is_pre_election: true
I20260928 23:07:12.672042 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f081a667c1641708c1e7b9a308d4fff" candidate_uuid: "71ded63256b44da292b61809e0eca1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" is_pre_election: true
I20260928 23:07:12.672088 24150 raft_consensus.cc:2454] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71ded63256b44da292b61809e0eca1bf in term 0.
I20260928 23:07:12.672142 23870 raft_consensus.cc:2454] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71ded63256b44da292b61809e0eca1bf in term 0.
I20260928 23:07:12.672251 23941 leader_election.cc:302] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.672374 24230 raft_consensus.cc:2781] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:12.672416 24230 raft_consensus.cc:482] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:12.672475 24230 raft_consensus.cc:3037] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.673321 24230 raft_consensus.cc:504] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.673448 24230 leader_election.cc:288] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [CANDIDATE]: Term 1 election: Requested vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843)
I20260928 23:07:12.673662 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f081a667c1641708c1e7b9a308d4fff" candidate_uuid: "71ded63256b44da292b61809e0eca1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
I20260928 23:07:12.673673 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f081a667c1641708c1e7b9a308d4fff" candidate_uuid: "71ded63256b44da292b61809e0eca1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c11e539122b410285b32fa6f9d31b81"
I20260928 23:07:12.673750 23870 raft_consensus.cc:3037] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.673763 24150 raft_consensus.cc:3037] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.674436 24150 raft_consensus.cc:2454] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71ded63256b44da292b61809e0eca1bf in term 1.
I20260928 23:07:12.674507 23870 raft_consensus.cc:2454] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71ded63256b44da292b61809e0eca1bf in term 1.
I20260928 23:07:12.674612 23941 leader_election.cc:302] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.674710 24230 raft_consensus.cc:2781] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:12.674835 24230 raft_consensus.cc:686] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 1 LEADER]: Becoming Leader. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Running, Role: LEADER
I20260928 23:07:12.674904 24230 consensus_queue.cc:237] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.675591 23713 catalog_manager.cc:5495] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 71ded63256b44da292b61809e0eca1bf (127.15.212.66). New cstate: current_term: 1 leader_uuid: "71ded63256b44da292b61809e0eca1bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
W20260928 23:07:12.687412 24197 tablet.cc:2369] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636831, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.126@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.712777 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.705138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44467 (local address 127.0.0.1:53362)
0928 23:07:12.705317 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.705329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.705410 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.705564 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.705565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.705958 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:12.706016 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.706019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.706682 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.706683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.707523 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.707526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.707952 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.707954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.708014 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.708617 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:12.708622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:12.710113 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:12.711767 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.711773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.711774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.711956 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.712170 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:12.712171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:12.712171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:12.712223 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:12.712519 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:12.712524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:12.712567 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.712624 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.712706 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260928 23:07:12.712890 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.705231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.15.212.126:44467)
0928 23:07:12.705282 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.705284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.705402 (+ 118us) server_negotiation.cc:407] Connection header received
0928 23:07:12.705473 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.705474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.705482 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.705551 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.706106 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.706596 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.707620 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.707881 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.707931 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.710201 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.710203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.710203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.710210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.711690 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.712031 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.712032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.712033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.712093 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.712283 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.712283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.712284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.712408 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.712528 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.712672 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.712769 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":26}
I20260928 23:07:12.715098 23713 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 23:07:12.715319 23713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:12.716739 23990 tablet_service.cc:1467] Processing CreateTablet for tablet c2a776b8d1504ec58f7b591198028687 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa454b1a3a7e4f549a35777d6bdccbcd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:07:12.716758 24130 tablet_service.cc:1467] Processing CreateTablet for tablet c2a776b8d1504ec58f7b591198028687 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa454b1a3a7e4f549a35777d6bdccbcd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:07:12.716900 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a776b8d1504ec58f7b591198028687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.716915 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a776b8d1504ec58f7b591198028687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.718183 24228 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: Bootstrap starting.
I20260928 23:07:12.718284 23850 tablet_service.cc:1467] Processing CreateTablet for tablet c2a776b8d1504ec58f7b591198028687 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa454b1a3a7e4f549a35777d6bdccbcd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 23:07:12.718416 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a776b8d1504ec58f7b591198028687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.718453 24225 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: Bootstrap starting.
I20260928 23:07:12.718604 24228 tablet_bootstrap.cc:654] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.719133 24225 tablet_bootstrap.cc:654] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.719841 24224 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: Bootstrap starting.
I20260928 23:07:12.719894 24225 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: No bootstrap required, opened a new log
I20260928 23:07:12.719934 24225 ts_tablet_manager.cc:1397] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:07:12.720078 24225 raft_consensus.cc:348] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.720152 24225 raft_consensus.cc:374] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.720175 24225 raft_consensus.cc:729] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.720217 24225 consensus_queue.cc:260] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.720253 24224 tablet_bootstrap.cc:654] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.720304 24225 ts_tablet_manager.cc:1428] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:12.720376 24228 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: No bootstrap required, opened a new log
I20260928 23:07:12.720450 24228 ts_tablet_manager.cc:1397] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:12.720415 24230 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.720506 24230 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.720791 24228 raft_consensus.cc:348] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.720861 24228 raft_consensus.cc:374] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.720880 24228 raft_consensus.cc:729] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.720920 24228 consensus_queue.cc:260] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.721014 24228 ts_tablet_manager.cc:1428] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 23:07:12.721076 24232 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.721174 24232 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.721427 24224 tablet_bootstrap.cc:492] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: No bootstrap required, opened a new log
I20260928 23:07:12.721486 24224 ts_tablet_manager.cc:1397] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:12.721647 24224 raft_consensus.cc:348] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.721719 24224 raft_consensus.cc:374] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.721750 24224 raft_consensus.cc:729] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.721791 24224 consensus_queue.cc:260] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.721906 24224 ts_tablet_manager.cc:1428] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.722060 24244 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.722121 24244 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.842500 24232 raft_consensus.cc:482] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:12.842587 24232 raft_consensus.cc:504] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.842770 24232 leader_election.cc:288] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:12.842988 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a776b8d1504ec58f7b591198028687" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" is_pre_election: true
I20260928 23:07:12.842988 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a776b8d1504ec58f7b591198028687" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf" is_pre_election: true
I20260928 23:07:12.843084 24009 raft_consensus.cc:2454] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:12.843084 23870 raft_consensus.cc:2454] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:12.843286 24082 leader_election.cc:302] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c918286f0504d71ba0ceca9c88f1ff9, 6c11e539122b410285b32fa6f9d31b81; no voters:
I20260928 23:07:12.843397 24232 raft_consensus.cc:2781] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:12.843477 24232 raft_consensus.cc:482] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:12.843505 24232 raft_consensus.cc:3037] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.844190 24232 raft_consensus.cc:504] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.844347 24232 leader_election.cc:288] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Requested vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:12.844475 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a776b8d1504ec58f7b591198028687" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
I20260928 23:07:12.844552 23870 raft_consensus.cc:3037] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.844538 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a776b8d1504ec58f7b591198028687" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf"
I20260928 23:07:12.844600 24009 raft_consensus.cc:3037] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:12.845332 24009 raft_consensus.cc:2454] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:12.845331 23870 raft_consensus.cc:2454] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:12.845500 24083 leader_election.cc:302] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:12.845595 24232 raft_consensus.cc:2781] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:12.845799 24232 raft_consensus.cc:686] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [term 1 LEADER]: Becoming Leader. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Running, Role: LEADER
I20260928 23:07:12.845880 24232 consensus_queue.cc:237] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.846244 24267 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: TxnStatusTablet state changed. Reason: New leader 6c11e539122b410285b32fa6f9d31b81. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.846361 24267 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:07:12.846541 24269 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 23:07:12.846663 24269 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 23:07:12.846625 23713 catalog_manager.cc:5495] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260928 23:07:12.868662 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.866216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.15.212.67:40843)
0928 23:07:12.866261 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.866263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.866272 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:12.866353 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.866354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.866375 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.866449 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:12.866612 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.867081 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.867835 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.868031 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.868114 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.868305 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.868403 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.868502 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.868567 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.868586 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:07:12.868669 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.866135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.0.0.1:53594)
0928 23:07:12.866206 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.866218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.866269 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.866466 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.866467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.866469 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:07:12.866527 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.866530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.867162 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.867163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.867753 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.867754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.868117 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.868118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.868182 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.868198 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.868496 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.868513 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.868553 (+ 40us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.868604 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:07:12.869800 24009 raft_consensus.cc:1261] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.869833 23870 raft_consensus.cc:1261] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.870024 24232 consensus_queue.cc:1035] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.870143 24267 consensus_queue.cc:1035] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.870692 24230 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: TxnStatusTablet state changed. Reason: New leader 6c11e539122b410285b32fa6f9d31b81. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.870785 24230 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.870746 24244 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: TxnStatusTablet state changed. Reason: New leader 6c11e539122b410285b32fa6f9d31b81. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.870819 24244 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.871598 24272 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.871650 24267 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.871692 24272 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:07:12.871699 24267 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 6c11e539122b410285b32fa6f9d31b81: This TxnStatusTablet replica's current role is: LEADER
I20260928 23:07:12.871977 24230 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.872054 24230 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 71ded63256b44da292b61809e0eca1bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.872704 24244 tablet_replica.cc:439] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } } }
I20260928 23:07:12.872781 24244 tablet_replica.cc:442] T c2a776b8d1504ec58f7b591198028687 P 0c918286f0504d71ba0ceca9c88f1ff9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 23:07:12.875892 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.873286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.15.212.66:41501)
0928 23:07:12.873327 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.873329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.873338 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:12.873370 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.873371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.873390 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.873464 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:12.873595 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.874341 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.875061 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.875367 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.875438 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.875582 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.875704 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.875776 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.875798 (+ 22us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.875817 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:07:12.881006 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.878258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:57005 (local address 127.15.212.67:40843)
0928 23:07:12.878297 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.878298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.878372 (+ 74us) server_negotiation.cc:407] Connection header received
0928 23:07:12.878441 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.878441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.878449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.878493 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:12.878621 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.879108 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.880193 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.880365 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.880498 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.880746 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.880783 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.880838 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.880896 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.880913 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:07:12.881021 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.878159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.15.212.66:57005)
0928 23:07:12.878323 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.878333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.878386 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.878505 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.878505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.878507 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:07:12.878563 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.878565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.879197 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.879198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.880120 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.880122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.880549 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.880555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.880606 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.880618 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.880851 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.880853 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.880895 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.880939 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260928 23:07:12.883829 24150 raft_consensus.cc:1261] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Refusing update from remote peer 71ded63256b44da292b61809e0eca1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.883898 23870 raft_consensus.cc:1261] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Refusing update from remote peer 71ded63256b44da292b61809e0eca1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:12.884089 24230 consensus_queue.cc:1035] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.884214 24256 consensus_queue.cc:1035] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:12.886992 24239 mvcc.cc:205] Tried to move back new op lower bound from 7334448467490578432 to 7334448466636894208. Current Snapshot: MvccSnapshot[applied={T|T < 7334448467490578432}]
I20260928 23:07:12.893180 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.890330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41501 (local address 127.15.212.67:57515)
0928 23:07:12.890396 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:12.890413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:12.890444 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:12.890678 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:12.890678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:12.890680 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:07:12.890728 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.890729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.891635 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.891636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.892217 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:12.892219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.892714 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.892715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:12.892783 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.892797 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.892957 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.892958 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:12.893039 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 23:07:12.893079 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:07:12.893420 24235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.890502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57515 (local address 127.15.212.66:41501)
0928 23:07:12.890567 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.890569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.890578 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:12.890599 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.890600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.890609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.890675 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:12.890767 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.891486 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.892296 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.892616 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.892681 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.892865 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:12.892892 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:12.892932 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.893336 (+ 404us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.893356 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:07:12.899655 23713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:12.899758 23713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:12.901193 23713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Sending DeleteTablet for 3 replicas of tablet 5f081a667c1641708c1e7b9a308d4fff
I20260928 23:07:12.901430 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f081a667c1641708c1e7b9a308d4fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53590
I20260928 23:07:12.901499 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f081a667c1641708c1e7b9a308d4fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:43664
I20260928 23:07:12.901616 24250 tablet_replica.cc:330] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: stopping tablet replica
I20260928 23:07:12.901628 24249 tablet_replica.cc:330] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: stopping tablet replica
I20260928 23:07:12.901672 24250 raft_consensus.cc:2229] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:12.901684 24249 raft_consensus.cc:2229] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:12.901711 24250 raft_consensus.cc:2258] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.901724 24249 raft_consensus.cc:2258] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.902021 23990 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f081a667c1641708c1e7b9a308d4fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:12 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36234
I20260928 23:07:12.902135 24251 tablet_replica.cc:330] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: stopping tablet replica
I20260928 23:07:12.902194 24251 raft_consensus.cc:2229] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:12.902275 24251 raft_consensus.cc:2258] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:12.902681 24251 ts_tablet_manager.cc:1905] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.904520 24251 ts_tablet_manager.cc:1918] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:07:12.904593 24251 log.cc:1196] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/5f081a667c1641708c1e7b9a308d4fff
I20260928 23:07:12.904619 24249 ts_tablet_manager.cc:1905] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.904806 24251 ts_tablet_manager.cc:1939] T 5f081a667c1641708c1e7b9a308d4fff P 71ded63256b44da292b61809e0eca1bf: Deleting consensus metadata
I20260928 23:07:12.905194 23699 catalog_manager.cc:4841] TS 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501): tablet 5f081a667c1641708c1e7b9a308d4fff (table test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]) successfully deleted
I20260928 23:07:12.905457 24250 ts_tablet_manager.cc:1905] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:12.906199 24249 ts_tablet_manager.cc:1918] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:07:12.906266 24249 log.cc:1196] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/5f081a667c1641708c1e7b9a308d4fff
I20260928 23:07:12.906482 24249 ts_tablet_manager.cc:1939] T 5f081a667c1641708c1e7b9a308d4fff P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting consensus metadata
I20260928 23:07:12.906896 23698 catalog_manager.cc:4841] TS 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417): tablet 5f081a667c1641708c1e7b9a308d4fff (table test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]) successfully deleted
I20260928 23:07:12.907042 24250 ts_tablet_manager.cc:1918] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:07:12.907099 24250 log.cc:1196] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/5f081a667c1641708c1e7b9a308d4fff
I20260928 23:07:12.907316 24250 ts_tablet_manager.cc:1939] T 5f081a667c1641708c1e7b9a308d4fff P 6c11e539122b410285b32fa6f9d31b81: Deleting consensus metadata
I20260928 23:07:12.907590 23697 catalog_manager.cc:4841] TS 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843): tablet 5f081a667c1641708c1e7b9a308d4fff (table test-table [id=112f3fc8bc774f0c86003d5f49c3e38e]) successfully deleted
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 23:07:12.927289 23763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:12.922074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.15.212.126:44467)
0928 23:07:12.922125 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.922128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.922263 (+ 135us) server_negotiation.cc:407] Connection header received
0928 23:07:12.922344 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.922345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.922355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.922412 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.922973 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.923468 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.924211 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.924411 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.924520 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.927243 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53374: BlockingRecv error: recv got EOF from 127.0.0.1:53374 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 28 23:07:12 dist-test-slave-x2t8 krb5kdc[23668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM
I20260928 23:07:12.935252 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:12.928268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.15.212.126:44467)
0928 23:07:12.928310 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:12.928312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:12.928720 (+ 408us) server_negotiation.cc:407] Connection header received
0928 23:07:12.928795 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:12.928796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:12.928806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:12.928853 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:12.929294 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.929753 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.930422 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:12.930627 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:12.930700 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:12.932810 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:12.932811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:12.932812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:12.932819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:12.934169 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.934517 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.934518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.934519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.934578 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:12.934771 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:12.934772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:12.934772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:12.934879 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:12.934978 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:12.935082 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 23:07:12.935154 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:07:12.936800 23713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:12.936971 23713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:12.938351 23850 tablet_service.cc:1467] Processing CreateTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 (DEFAULT_TABLE table=test-table [id=a7a977c208cc4e85a11722e37baaad3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.938444 23990 tablet_service.cc:1467] Processing CreateTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 (DEFAULT_TABLE table=test-table [id=a7a977c208cc4e85a11722e37baaad3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.938489 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77cbfdae8b4d5e980d5408bbb9d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.938544 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77cbfdae8b4d5e980d5408bbb9d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.939802 24130 tablet_service.cc:1467] Processing CreateTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 (DEFAULT_TABLE table=test-table [id=a7a977c208cc4e85a11722e37baaad3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:12.939908 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77cbfdae8b4d5e980d5408bbb9d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:12.941231 24224 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Bootstrap starting.
I20260928 23:07:12.941531 24224 tablet_bootstrap.cc:654] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.941776 24225 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Bootstrap starting.
I20260928 23:07:12.942135 24225 tablet_bootstrap.cc:654] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.942451 24228 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Bootstrap starting.
I20260928 23:07:12.942790 24228 tablet_bootstrap.cc:654] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:12.943753 24225 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: No bootstrap required, opened a new log
I20260928 23:07:12.943786 24224 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: No bootstrap required, opened a new log
I20260928 23:07:12.943799 24225 ts_tablet_manager.cc:1397] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:12.943826 24224 ts_tablet_manager.cc:1397] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 23:07:12.943939 24224 raft_consensus.cc:348] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.943940 24225 raft_consensus.cc:348] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.944000 24225 raft_consensus.cc:374] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.944000 24224 raft_consensus.cc:374] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.944020 24224 raft_consensus.cc:729] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.944020 24225 raft_consensus.cc:729] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.944060 24225 consensus_queue.cc:260] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.944072 24224 consensus_queue.cc:260] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.944142 24225 ts_tablet_manager.cc:1428] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.944154 24224 ts_tablet_manager.cc:1428] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:12.945195 24228 tablet_bootstrap.cc:492] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: No bootstrap required, opened a new log
I20260928 23:07:12.945245 24228 ts_tablet_manager.cc:1397] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 23:07:12.945389 24228 raft_consensus.cc:348] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.945451 24228 raft_consensus.cc:374] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:12.945464 24228 raft_consensus.cc:729] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Initialized, Role: FOLLOWER
I20260928 23:07:12.945518 24228 consensus_queue.cc:260] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:12.945605 24228 ts_tablet_manager.cc:1428] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:13.228130 24267 raft_consensus.cc:482] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:13.228217 24267 raft_consensus.cc:504] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.228374 24267 leader_election.cc:288] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:13.228621 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77cbfdae8b4d5e980d5408bbb9d600" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" is_pre_election: true
I20260928 23:07:13.228703 23870 raft_consensus.cc:2454] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:13.228638 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77cbfdae8b4d5e980d5408bbb9d600" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf" is_pre_election: true
I20260928 23:07:13.228777 24009 raft_consensus.cc:2454] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:13.228875 24082 leader_election.cc:302] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c918286f0504d71ba0ceca9c88f1ff9, 6c11e539122b410285b32fa6f9d31b81; no voters:
I20260928 23:07:13.228981 24267 raft_consensus.cc:2781] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:13.229017 24267 raft_consensus.cc:482] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:13.229038 24267 raft_consensus.cc:3037] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.229769 24267 raft_consensus.cc:504] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.229911 24267 leader_election.cc:288] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Requested vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:13.230036 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77cbfdae8b4d5e980d5408bbb9d600" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf"
I20260928 23:07:13.230100 24009 raft_consensus.cc:3037] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.230049 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77cbfdae8b4d5e980d5408bbb9d600" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
I20260928 23:07:13.230131 23870 raft_consensus.cc:3037] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.230751 24009 raft_consensus.cc:2454] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:13.230906 23870 raft_consensus.cc:2454] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:13.230906 24083 leader_election.cc:302] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:13.231010 24267 raft_consensus.cc:2781] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:13.231068 24267 raft_consensus.cc:686] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 1 LEADER]: Becoming Leader. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Running, Role: LEADER
I20260928 23:07:13.231118 24267 consensus_queue.cc:237] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.231654 23712 catalog_manager.cc:5495] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260928 23:07:13.297765 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:13.295059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.15.212.67:40843)
0928 23:07:13.295120 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:13.295121 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:13.295132 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:07:13.295182 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:13.295183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:13.295193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:13.295248 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:13.295392 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.295863 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.296757 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.297130 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.297212 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:13.297419 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:13.297456 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:13.297514 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:13.297631 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 23:07:13.297652 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 23:07:13.298527 24009 raft_consensus.cc:1261] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:13.298548 23870 raft_consensus.cc:1261] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:13.298754 24267 consensus_queue.cc:1035] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:13.298907 24232 consensus_queue.cc:1035] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:13.301754 23713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:13.301842 23713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:13.302457 23713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Sending DeleteTablet for 3 replicas of tablet bd77cbfdae8b4d5e980d5408bbb9d600
I20260928 23:07:13.302717 23990 tablet_service.cc:1514] Processing DeleteTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36234
I20260928 23:07:13.302739 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:43664
I20260928 23:07:13.302747 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet bd77cbfdae8b4d5e980d5408bbb9d600 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53590
I20260928 23:07:13.302807 24251 tablet_replica.cc:330] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: stopping tablet replica
I20260928 23:07:13.302819 24250 tablet_replica.cc:330] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: stopping tablet replica
I20260928 23:07:13.302822 24249 tablet_replica.cc:330] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: stopping tablet replica
I20260928 23:07:13.302860 24251 raft_consensus.cc:2229] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:13.302875 24250 raft_consensus.cc:2229] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:13.302881 24249 raft_consensus.cc:2229] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:13.302901 24251 raft_consensus.cc:2258] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.302918 24249 raft_consensus.cc:2258] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.302942 24250 raft_consensus.cc:2258] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.303259 24251 ts_tablet_manager.cc:1905] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:13.303335 24250 ts_tablet_manager.cc:1905] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:13.303399 24249 ts_tablet_manager.cc:1905] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:13.304878 23713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:13.305012 23713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:13.305190 24249 ts_tablet_manager.cc:1918] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:13.305245 24249 log.cc:1196] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/bd77cbfdae8b4d5e980d5408bbb9d600
I20260928 23:07:13.305238 24251 ts_tablet_manager.cc:1918] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:13.305267 24250 ts_tablet_manager.cc:1918] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:13.305289 24251 log.cc:1196] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/bd77cbfdae8b4d5e980d5408bbb9d600
I20260928 23:07:13.305316 24250 log.cc:1196] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/bd77cbfdae8b4d5e980d5408bbb9d600
I20260928 23:07:13.305519 24249 ts_tablet_manager.cc:1939] T bd77cbfdae8b4d5e980d5408bbb9d600 P 0c918286f0504d71ba0ceca9c88f1ff9: Deleting consensus metadata
I20260928 23:07:13.305523 24251 ts_tablet_manager.cc:1939] T bd77cbfdae8b4d5e980d5408bbb9d600 P 71ded63256b44da292b61809e0eca1bf: Deleting consensus metadata
I20260928 23:07:13.305523 24250 ts_tablet_manager.cc:1939] T bd77cbfdae8b4d5e980d5408bbb9d600 P 6c11e539122b410285b32fa6f9d31b81: Deleting consensus metadata
I20260928 23:07:13.305863 23698 catalog_manager.cc:4841] TS 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417): tablet bd77cbfdae8b4d5e980d5408bbb9d600 (table test-table [id=a7a977c208cc4e85a11722e37baaad3d]) successfully deleted
I20260928 23:07:13.305860 23697 catalog_manager.cc:4841] TS 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843): tablet bd77cbfdae8b4d5e980d5408bbb9d600 (table test-table [id=a7a977c208cc4e85a11722e37baaad3d]) successfully deleted
I20260928 23:07:13.305974 23699 catalog_manager.cc:4841] TS 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501): tablet bd77cbfdae8b4d5e980d5408bbb9d600 (table test-table [id=a7a977c208cc4e85a11722e37baaad3d]) successfully deleted
I20260928 23:07:13.306838 23990 tablet_service.cc:1467] Processing CreateTablet for tablet c7d6409b60014126a10fd8b9473ed855 (DEFAULT_TABLE table=test-table [id=52cf837671744841aba5a6ba023c9da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:13.306829 23850 tablet_service.cc:1467] Processing CreateTablet for tablet c7d6409b60014126a10fd8b9473ed855 (DEFAULT_TABLE table=test-table [id=52cf837671744841aba5a6ba023c9da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:13.306885 24130 tablet_service.cc:1467] Processing CreateTablet for tablet c7d6409b60014126a10fd8b9473ed855 (DEFAULT_TABLE table=test-table [id=52cf837671744841aba5a6ba023c9da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:13.306929 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d6409b60014126a10fd8b9473ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:13.306928 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d6409b60014126a10fd8b9473ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:13.306979 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d6409b60014126a10fd8b9473ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:13.308024 24224 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: Bootstrap starting.
I20260928 23:07:13.308141 24225 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Bootstrap starting.
I20260928 23:07:13.308319 24224 tablet_bootstrap.cc:654] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:13.308473 24225 tablet_bootstrap.cc:654] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:13.308702 24228 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Bootstrap starting.
I20260928 23:07:13.308979 24224 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: No bootstrap required, opened a new log
I20260928 23:07:13.308977 24228 tablet_bootstrap.cc:654] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:13.309042 24224 ts_tablet_manager.cc:1397] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:07:13.309154 24224 raft_consensus.cc:348] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.309242 24224 raft_consensus.cc:374] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:13.309269 24224 raft_consensus.cc:729] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c918286f0504d71ba0ceca9c88f1ff9, State: Initialized, Role: FOLLOWER
I20260928 23:07:13.309310 24224 consensus_queue.cc:260] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.309381 24225 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: No bootstrap required, opened a new log
I20260928 23:07:13.309394 24224 ts_tablet_manager.cc:1428] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:13.309433 24225 ts_tablet_manager.cc:1397] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:07:13.309540 24225 raft_consensus.cc:348] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.309608 24225 raft_consensus.cc:374] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:13.309629 24225 raft_consensus.cc:729] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ded63256b44da292b61809e0eca1bf, State: Initialized, Role: FOLLOWER
I20260928 23:07:13.309672 24225 consensus_queue.cc:260] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.309759 24225 ts_tablet_manager.cc:1428] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:13.310431 24228 tablet_bootstrap.cc:492] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: No bootstrap required, opened a new log
I20260928 23:07:13.310496 24228 ts_tablet_manager.cc:1397] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:07:13.310613 24228 raft_consensus.cc:348] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.310683 24228 raft_consensus.cc:374] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:13.310716 24228 raft_consensus.cc:729] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Initialized, Role: FOLLOWER
I20260928 23:07:13.310763 24228 consensus_queue.cc:260] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.310850 24228 ts_tablet_manager.cc:1428] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:13.479002 24267 raft_consensus.cc:482] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:13.479094 24267 raft_consensus.cc:504] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.479270 24267 leader_election.cc:288] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:13.479497 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d6409b60014126a10fd8b9473ed855" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" is_pre_election: true
I20260928 23:07:13.479501 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d6409b60014126a10fd8b9473ed855" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf" is_pre_election: true
I20260928 23:07:13.479609 24009 raft_consensus.cc:2454] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:13.479609 23870 raft_consensus.cc:2454] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 0.
I20260928 23:07:13.479776 24083 leader_election.cc:302] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:13.479899 24267 raft_consensus.cc:2781] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:13.479976 24267 raft_consensus.cc:482] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:13.479998 24267 raft_consensus.cc:3037] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.480729 24267 raft_consensus.cc:504] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.480867 24267 leader_election.cc:288] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Requested vote from peers 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417), 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501)
I20260928 23:07:13.481015 23870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d6409b60014126a10fd8b9473ed855" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c918286f0504d71ba0ceca9c88f1ff9"
I20260928 23:07:13.481041 24009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d6409b60014126a10fd8b9473ed855" candidate_uuid: "6c11e539122b410285b32fa6f9d31b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71ded63256b44da292b61809e0eca1bf"
I20260928 23:07:13.481117 24009 raft_consensus.cc:3037] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.481120 23870 raft_consensus.cc:3037] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:13.481755 24009 raft_consensus.cc:2454] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:13.481910 24083 leader_election.cc:302] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c11e539122b410285b32fa6f9d31b81, 71ded63256b44da292b61809e0eca1bf; no voters:
I20260928 23:07:13.481974 23870 raft_consensus.cc:2454] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c11e539122b410285b32fa6f9d31b81 in term 1.
I20260928 23:07:13.481992 24267 raft_consensus.cc:2781] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:13.482036 24267 raft_consensus.cc:686] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 1 LEADER]: Becoming Leader. State: Replica: 6c11e539122b410285b32fa6f9d31b81, State: Running, Role: LEADER
I20260928 23:07:13.482084 24267 consensus_queue.cc:237] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } }
I20260928 23:07:13.482610 23713 catalog_manager.cc:5495] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "6c11e539122b410285b32fa6f9d31b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c11e539122b410285b32fa6f9d31b81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260928 23:07:13.495921 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:13.493708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:33331 (local address 127.15.212.67:40843)
0928 23:07:13.493776 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:13.493778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:13.493840 (+ 62us) server_negotiation.cc:407] Connection header received
0928 23:07:13.493895 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:13.493896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:13.493903 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:13.493991 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:13.494148 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.494583 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.495392 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.495561 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.495613 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:13.495725 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:13.495748 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:13.495785 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:13.495834 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 23:07:13.495854 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":46}
I20260928 23:07:13.495991 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:13.493590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40843 (local address 127.15.212.67:33331)
0928 23:07:13.493771 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:13.493780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:13.493848 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:13.494016 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:13.494017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:13.494019 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 23:07:13.494074 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:13.494076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.494661 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.494662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:13.495326 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:13.495327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:13.495621 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:13.495622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:13.495668 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:13.495672 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 23:07:13.495800 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 23:07:13.495801 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:07:13.495834 (+ 33us) client_negotiation.cc:240] Negotiation successful
0928 23:07:13.495873 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260928 23:07:13.497803 23870 raft_consensus.cc:1261] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:13.497816 24009 raft_consensus.cc:1261] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Refusing update from remote peer 6c11e539122b410285b32fa6f9d31b81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:13.498000 24267 consensus_queue.cc:1035] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71ded63256b44da292b61809e0eca1bf" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:13.498085 24232 consensus_queue.cc:1035] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c918286f0504d71ba0ceca9c88f1ff9" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 43417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:13.507419 23711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:13.507513 23711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:13.508033 23711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76204f84b5284b5e9647db1c4e747fc7: Sending DeleteTablet for 3 replicas of tablet c7d6409b60014126a10fd8b9473ed855
I20260928 23:07:13.508286 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d6409b60014126a10fd8b9473ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:43664
I20260928 23:07:13.508287 23990 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d6409b60014126a10fd8b9473ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36234
I20260928 23:07:13.508373 24251 tablet_replica.cc:330] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: stopping tablet replica
I20260928 23:07:13.508420 24251 raft_consensus.cc:2229] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:13.508416 24249 tablet_replica.cc:330] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9: stopping tablet replica
I20260928 23:07:13.508481 24251 raft_consensus.cc:2258] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.508512 24249 raft_consensus.cc:2229] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:13.508553 24249 raft_consensus.cc:2258] T c7d6409b60014126a10fd8b9473ed855 P 0c918286f0504d71ba0ceca9c88f1ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.508599 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23785
I20260928 23:07:13.509341 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet c7d6409b60014126a10fd8b9473ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:13 UTC) from {username='oryx', principal='oryx/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53590
I20260928 23:07:13.509421 24250 tablet_replica.cc:330] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: stopping tablet replica
I20260928 23:07:13.509470 24250 raft_consensus.cc:2229] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:13.509526 24250 raft_consensus.cc:2258] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:13.509845 24250 ts_tablet_manager.cc:1905] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:13.509910 24251 ts_tablet_manager.cc:1905] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:13.511340 24251 ts_tablet_manager.cc:1918] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:07:13.511396 24251 log.cc:1196] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/c7d6409b60014126a10fd8b9473ed855
I20260928 23:07:13.511617 24251 ts_tablet_manager.cc:1939] T c7d6409b60014126a10fd8b9473ed855 P 71ded63256b44da292b61809e0eca1bf: Deleting consensus metadata
I20260928 23:07:13.511708 24250 ts_tablet_manager.cc:1918] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 23:07:13.511770 24250 log.cc:1196] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/c7d6409b60014126a10fd8b9473ed855
I20260928 23:07:13.511911 23699 catalog_manager.cc:4841] TS 71ded63256b44da292b61809e0eca1bf (127.15.212.66:41501): tablet c7d6409b60014126a10fd8b9473ed855 (table test-table [id=52cf837671744841aba5a6ba023c9da2]) successfully deleted
I20260928 23:07:13.512187 24250 ts_tablet_manager.cc:1939] T c7d6409b60014126a10fd8b9473ed855 P 6c11e539122b410285b32fa6f9d31b81: Deleting consensus metadata
I20260928 23:07:13.512532 23697 catalog_manager.cc:4841] TS 6c11e539122b410285b32fa6f9d31b81 (127.15.212.67:40843): tablet c7d6409b60014126a10fd8b9473ed855 (table test-table [id=52cf837671744841aba5a6ba023c9da2]) successfully deleted
W20260928 23:07:13.515829 23698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:43417 (error 108)
I20260928 23:07:13.516043 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23925
I20260928 23:07:13.520620 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:13.516164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:43417 (local address 127.0.0.1:43682)
0928 23:07:13.520532 (+ 4368us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:13.520589 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4349,"thread_start_us":35,"threads_started":1}
W20260928 23:07:13.520761 23698 catalog_manager.cc:4584] TS 0c918286f0504d71ba0ceca9c88f1ff9 (127.15.212.65:43417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7d6409b60014126a10fd8b9473ed855: Network error: Client connection negotiation failed: client connection to 127.15.212.65:43417: connect: Connection refused (error 111)
I20260928 23:07:13.521399 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24065
I20260928 23:07:13.527269 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 23681
2026-09-28T23:07:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2852 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:13 dist-test-slave-x2t8 krb5kdc[24317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:13 dist-test-slave-x2t8 krb5kdc[24317](info): set up 2 sockets
Sep 28 23:07:13 dist-test-slave-x2t8 krb5kdc[24317](info): commencing operation
krb5kdc: starting...
W20260928 23:07:14.552896 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:14 dist-test-slave-x2t8 krb5kdc[24317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:07:14.666769 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34183
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:39869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:14.744568 24330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:14.744771 24330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:14.744809 24330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:14.745923 24330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:14.745962 24330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:14.745975 24330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:14.745988 24330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:14.746039 24330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:14.747427 24330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:14.747668 24330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:14.747854 24330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:14.752570 24338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:14.752549 24336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:14.752561 24335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:14.752650 24330 server_base.cc:1034] running on GCE node
I20260928 23:07:14.753036 24330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:14.753310 24330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:14.754484 24330 hybrid_clock.cc:648] HybridClock initialized: now 1790636834754467 us; error 41 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T23:07:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1236 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:14 dist-test-slave-x2t8 krb5kdc[24343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:14 dist-test-slave-x2t8 krb5kdc[24343](info): set up 2 sockets
Sep 28 23:07:14 dist-test-slave-x2t8 krb5kdc[24343](info): commencing operation
krb5kdc: starting...
W20260928 23:07:15.788941 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:15 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:07:15.902866 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46143
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:15.979036 24356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:15.979250 24356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:15.979274 24356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:15.980417 24356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:15.980506 24356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:15.980530 24356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:15.980551 24356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:15.980581 24356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:15.981988 24356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:15.982257 24356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:15.982470 24356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:15.987257 24364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:15.987264 24361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:15.987268 24362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:15.987612 24356 server_base.cc:1034] running on GCE node
I20260928 23:07:15.987825 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:15.988085 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:15.989240 24356 hybrid_clock.cc:648] HybridClock initialized: now 1790636835989217 us; error 34 us; skew 500 ppm
Sep 28 23:07:15 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636835, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:15.997419 24356 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:07:15.997888 24356 webserver.cc:460] Webserver started at http://127.15.212.126:39361/ using document root <none> and password file <none>
I20260928 23:07:15.998122 24356 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:15.998190 24356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:15.998338 24356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:15.999344 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "68d3373d360e4d9cb9c99aae38df53d9"
format_stamp: "Formatted at 2026-09-28 23:07:15 on dist-test-slave-x2t8"
server_key: "699e7d097629a11ed7862ab21880dd01"
server_key_iv: "e10d03e38380302f05a3db4d4458b024"
server_key_version: "encryptionkey@0"
I20260928 23:07:15.999694 24356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "68d3373d360e4d9cb9c99aae38df53d9"
format_stamp: "Formatted at 2026-09-28 23:07:15 on dist-test-slave-x2t8"
server_key: "699e7d097629a11ed7862ab21880dd01"
server_key_iv: "e10d03e38380302f05a3db4d4458b024"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.001178 24356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:16.001929 24371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.002138 24356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.002249 24356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "68d3373d360e4d9cb9c99aae38df53d9"
format_stamp: "Formatted at 2026-09-28 23:07:15 on dist-test-slave-x2t8"
server_key: "699e7d097629a11ed7862ab21880dd01"
server_key_iv: "e10d03e38380302f05a3db4d4458b024"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.002326 24356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.011492 24356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.013649 24356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.013804 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.017812 24356 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46143
I20260928 23:07:16.017856 24423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46143 every 8 connection(s)
I20260928 23:07:16.018186 24356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:16.019029 24424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:16.021350 24424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Bootstrap starting.
I20260928 23:07:16.021901 24424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:16.022204 24424 log.cc:824] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:16.022949 24424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: No bootstrap required, opened a new log
I20260928 23:07:16.024578 24424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.024709 24424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:16.024731 24424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d3373d360e4d9cb9c99aae38df53d9, State: Initialized, Role: FOLLOWER
I20260928 23:07:16.024887 24424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.024951 24424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:16.024973 24424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:16.025045 24424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:16.025534 24424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.025640 24424 leader_election.cc:302] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68d3373d360e4d9cb9c99aae38df53d9; no voters:
I20260928 23:07:16.025822 24424 leader_election.cc:288] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:16.025857 24427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:16.026051 24427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 LEADER]: Becoming Leader. State: Replica: 68d3373d360e4d9cb9c99aae38df53d9, State: Running, Role: LEADER
I20260928 23:07:16.026109 24424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:16.026186 24427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.026544 24429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68d3373d360e4d9cb9c99aae38df53d9. Latest consensus state: current_term: 1 leader_uuid: "68d3373d360e4d9cb9c99aae38df53d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } } }
I20260928 23:07:16.026661 24428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68d3373d360e4d9cb9c99aae38df53d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } } }
I20260928 23:07:16.026692 24429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:16.026772 24428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:16.027050 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24356
I20260928 23:07:16.027155 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:16.027331 16209 external_mini_cluster.cc:1428] Setting key 43b457235c038b34fdac009832aaf72b
I20260928 23:07:16.027752 24436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:16.028278 24436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:16.029857 24436 catalog_manager.cc:1347] Generated new cluster ID: 776b62de77934a479242058d835ba614
I20260928 23:07:16.029912 24436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:16.038679 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.027991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43878 (local address 127.15.212.126:46143)
0928 23:07:16.028163 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.028174 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.028210 (+ 36us) server_negotiation.cc:407] Connection header received
0928 23:07:16.028333 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.028342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.028404 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.029253 (+ 849us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.029387 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.030674 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.031756 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.032325 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.032414 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.035163 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:16.035166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:16.035167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:16.035178 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:16.037065 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.037432 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.037434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.037435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.037587 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.037801 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.037802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.037802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.038077 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:16.038217 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:16.038331 (+ 114us) server_negotiation.cc:299] Negotiation successful
0928 23:07:16.038460 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260928 23:07:16.039672 24436 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:16.040166 24436 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:16.042420 24436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Generated new TSK 0
I20260928 23:07:16.042577 24436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:07:16.070853 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:16.186264 24450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.186532 24450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.186568 24450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.188272 24450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.188333 24450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.188370 24450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:16.190320 24450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.190546 24450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.190755 24450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:16.195811 24458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.195814 24455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.195791 24456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.195926 24450 server_base.cc:1034] running on GCE node
I20260928 23:07:16.196285 24450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:16.196589 24450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:16.197757 24450 hybrid_clock.cc:648] HybridClock initialized: now 1790636836197743 us; error 33 us; skew 500 ppm
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:16.200224 24450 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:07:16.200700 24450 webserver.cc:460] Webserver started at http://127.15.212.65:37327/ using document root <none> and password file <none>
I20260928 23:07:16.200922 24450 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:16.200994 24450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:16.201125 24450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:16.202104 24450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "cc12304e4c8d4444b3bc7dbb2e71c771"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "8598650b355f58580bad91b74fa0390e"
server_key_iv: "20624ae2b29526bf2445f8d95c9fed8f"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.202450 24450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "cc12304e4c8d4444b3bc7dbb2e71c771"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "8598650b355f58580bad91b74fa0390e"
server_key_iv: "20624ae2b29526bf2445f8d95c9fed8f"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.203775 24450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:16.204581 24465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.204748 24450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.204876 24450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "cc12304e4c8d4444b3bc7dbb2e71c771"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "8598650b355f58580bad91b74fa0390e"
server_key_iv: "20624ae2b29526bf2445f8d95c9fed8f"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.204967 24450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.219101 24450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.220908 24450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.221065 24450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.221280 24450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:16.221592 24450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:16.221647 24450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.221688 24450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:16.221714 24450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.227753 24450 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:37935
I20260928 23:07:16.227829 24578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:37935 every 8 connection(s)
I20260928 23:07:16.228166 24450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:16.234006 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24450
I20260928 23:07:16.234102 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:16.234300 16209 external_mini_cluster.cc:1428] Setting key afb24f211f7572722187bb9d658a1324
I20260928 23:07:16.237581 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.228892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:43911 (local address 127.15.212.126:46143)
0928 23:07:16.228954 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.228956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.229146 (+ 190us) server_negotiation.cc:407] Connection header received
0928 23:07:16.229588 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.229589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.229601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.229657 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.230483 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.230964 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.231868 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.232056 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.232107 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.234379 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:16.234381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:16.234381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:16.234390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:16.236062 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.236612 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.236614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.236614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.236682 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.236917 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.236918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.236918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.237057 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:16.237126 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:16.237361 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 23:07:16.237456 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:07:16.237645 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.228862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:43911)
0928 23:07:16.229022 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.229061 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.229510 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.229707 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.229710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.230239 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.230411 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.230416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.231039 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.231039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.231799 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.231800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.232127 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.232128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.232221 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.232777 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.232785 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.234258 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:16.236146 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.236153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.236156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.236510 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.236748 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.236749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.236749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.236849 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.237122 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:16.237126 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:16.237234 (+ 108us) client_negotiation.cc:769] Sending connection context
0928 23:07:16.237313 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:07:16.237423 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260928 23:07:16.238139 24579 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46143
I20260928 23:07:16.238230 24579 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:16.238397 24579 heartbeater.cc:511] Master 127.15.212.126:46143 requested a full tablet report, sending...
I20260928 23:07:16.238933 24388 ts_manager.cc:194] Registered new tserver with Master: cc12304e4c8d4444b3bc7dbb2e71c771 (127.15.212.65:37935)
I20260928 23:07:16.239806 24388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:43911
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:07:16.258716 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:16.337774 24584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.337996 24584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.338032 24584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.339175 24584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.339237 24584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.339282 24584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:16.340750 24584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.341010 24584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.341231 24584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:16.346056 24592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.346122 24590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.346069 24589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.346362 24584 server_base.cc:1034] running on GCE node
I20260928 23:07:16.346555 24584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:16.346796 24584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:16.347991 24584 hybrid_clock.cc:648] HybridClock initialized: now 1790636836347967 us; error 34 us; skew 500 ppm
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:16.350363 24584 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:07:16.350791 24584 webserver.cc:460] Webserver started at http://127.15.212.66:41675/ using document root <none> and password file <none>
I20260928 23:07:16.351024 24584 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:16.351095 24584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:16.351243 24584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:16.352299 24584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "28cdcb9f036c4beab840c531d0951a05"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "4a91fc88e396fdadae93a517ac6eca05"
server_key_iv: "34c5c72d8df3e2c0476a50d2aeb77a4d"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.352705 24584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "28cdcb9f036c4beab840c531d0951a05"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "4a91fc88e396fdadae93a517ac6eca05"
server_key_iv: "34c5c72d8df3e2c0476a50d2aeb77a4d"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.354001 24584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.354705 24599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.354872 24584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:16.354969 24584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "28cdcb9f036c4beab840c531d0951a05"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "4a91fc88e396fdadae93a517ac6eca05"
server_key_iv: "34c5c72d8df3e2c0476a50d2aeb77a4d"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.355063 24584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.367457 24584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.369207 24584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.369365 24584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.369577 24584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:16.369874 24584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:16.369930 24584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.369971 24584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:16.369997 24584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.375519 24584 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:38089
I20260928 23:07:16.375586 24712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:38089 every 8 connection(s)
I20260928 23:07:16.375905 24584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:16.382810 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24584
I20260928 23:07:16.382927 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:16.383137 16209 external_mini_cluster.cc:1428] Setting key 60bbd6a2c9bcd78784b98f3d8644e02f
I20260928 23:07:16.385288 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.376666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:51695 (local address 127.15.212.126:46143)
0928 23:07:16.376737 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.376740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.376937 (+ 197us) server_negotiation.cc:407] Connection header received
0928 23:07:16.377416 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.377417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.377429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.377498 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.378376 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.378880 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.379862 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.380067 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.380157 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.382181 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:16.382183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:16.382184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:16.382191 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:16.383747 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.384234 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.384235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.384235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.384302 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.384639 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.384640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.384641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.384761 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:16.384840 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:16.385115 (+ 275us) server_negotiation.cc:299] Negotiation successful
0928 23:07:16.385203 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:07:16.385457 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.376643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:51695)
0928 23:07:16.376811 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.376845 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.377320 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.377570 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.377574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.378101 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.378279 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.378283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.378974 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.378975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.379755 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.379757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.380164 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.380165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.380283 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.380795 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.380802 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.382086 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:16.383936 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.383941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.383942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.384152 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.384371 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.384372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.384372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.384567 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.384829 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:16.384835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:16.384971 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 23:07:16.385079 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 23:07:16.385227 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260928 23:07:16.385988 24713 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46143
I20260928 23:07:16.386081 24713 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:16.386245 24713 heartbeater.cc:511] Master 127.15.212.126:46143 requested a full tablet report, sending...
I20260928 23:07:16.386651 24388 ts_manager.cc:194] Registered new tserver with Master: 28cdcb9f036c4beab840c531d0951a05 (127.15.212.66:38089)
I20260928 23:07:16.387198 24388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:51695
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:07:16.408550 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:16.486657 24718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.486840 24718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.486861 24718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.487984 24718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.488024 24718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.488050 24718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:16.489434 24718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.489669 24718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.489885 24718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:16.494843 24726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.494858 24723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.494855 24724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.495118 24718 server_base.cc:1034] running on GCE node
I20260928 23:07:16.495304 24718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:16.495517 24718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:16.496655 24718 hybrid_clock.cc:648] HybridClock initialized: now 1790636836496644 us; error 27 us; skew 500 ppm
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:16.499202 24718 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:07:16.499557 24718 webserver.cc:460] Webserver started at http://127.15.212.67:44875/ using document root <none> and password file <none>
I20260928 23:07:16.499738 24718 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:16.499775 24718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:16.499867 24718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:16.500849 24718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "4415cbedb9bf4a2d8d9df11c8a72feb8"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "461b97fb8fdc20b9fd6561df353b8525"
server_key_iv: "92b6bf798632eb3a850dbf5137503fd3"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.501250 24718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "4415cbedb9bf4a2d8d9df11c8a72feb8"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "461b97fb8fdc20b9fd6561df353b8525"
server_key_iv: "92b6bf798632eb3a850dbf5137503fd3"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.502559 24718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:16.503237 24733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.503402 24718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:16.503499 24718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "4415cbedb9bf4a2d8d9df11c8a72feb8"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "461b97fb8fdc20b9fd6561df353b8525"
server_key_iv: "92b6bf798632eb3a850dbf5137503fd3"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.503582 24718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.517220 24718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.518854 24718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.519006 24718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.519222 24718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:16.519500 24718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:16.519554 24718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.519593 24718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:16.519619 24718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.525331 24718 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:33479
I20260928 23:07:16.525369 24846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:33479 every 8 connection(s)
I20260928 23:07:16.525650 24718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:16.532992 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24718
I20260928 23:07:16.533119 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:16.533313 16209 external_mini_cluster.cc:1428] Setting key 6c31bdd1a5f60a93d74f4bf51f11af0f
I20260928 23:07:16.534747 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.526303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:54989 (local address 127.15.212.126:46143)
0928 23:07:16.526359 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.526361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.526573 (+ 212us) server_negotiation.cc:407] Connection header received
0928 23:07:16.527060 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.527061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.527072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.527149 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.528084 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.528741 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.529705 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.529909 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.530004 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.531921 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:16.531922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:16.531922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:16.531930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:16.533284 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.533783 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.533784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.533785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.533855 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:16.534132 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:16.534133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:16.534134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:16.534270 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:16.534326 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:16.534589 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 23:07:16.534662 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:07:16.534935 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.526251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:54989)
0928 23:07:16.526430 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.526501 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.526957 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.527225 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.527229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.527819 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.527988 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.527992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.528818 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.528819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.529599 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.529601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.530011 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.530012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.530149 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.530586 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.530594 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.531827 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:16.533371 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.533377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.533379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.533700 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.533929 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:16.533930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:16.533930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:16.534041 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:16.534330 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:16.534335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:16.534460 (+ 125us) client_negotiation.cc:769] Sending connection context
0928 23:07:16.534570 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 23:07:16.534707 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260928 23:07:16.535285 24847 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46143
I20260928 23:07:16.535353 24847 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:16.535530 24847 heartbeater.cc:511] Master 127.15.212.126:46143 requested a full tablet report, sending...
I20260928 23:07:16.535872 24388 ts_manager.cc:194] Registered new tserver with Master: 4415cbedb9bf4a2d8d9df11c8a72feb8 (127.15.212.67:33479)
I20260928 23:07:16.536235 24388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:54989
I20260928 23:07:16.545035 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
Entry for principal oryx/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab.
WARNING: no policy specified for oryx/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
Entry for principal oryx/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab.
WARNING: no policy specified for oryx/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
Entry for principal oryx/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab.
I20260928 23:07:16.614826 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24450
I20260928 23:07:16.619350 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24584
I20260928 23:07:16.623437 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24718
I20260928 23:07:16.627780 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24356
I20260928 23:07:16.632766 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46143
--webserver_interface=127.15.212.126
--webserver_port=39361
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:16.710759 24856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.710989 24856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.711026 24856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.712152 24856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:16.712215 24856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.712239 24856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.712260 24856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:16.712288 24856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:16.713757 24856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=39361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.714016 24856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.714231 24856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:16.718890 24862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.718897 24864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.718883 24861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.719158 24856 server_base.cc:1034] running on GCE node
I20260928 23:07:16.719376 24856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:16.719600 24856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:16.720736 24856 hybrid_clock.cc:648] HybridClock initialized: now 1790636836720718 us; error 34 us; skew 500 ppm
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:16.722996 24856 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:07:16.723414 24856 webserver.cc:460] Webserver started at http://127.15.212.126:39361/ using document root <none> and password file <none>
I20260928 23:07:16.723592 24856 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:16.723651 24856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:16.725051 24856 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.725713 24871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.725903 24856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.725980 24856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "68d3373d360e4d9cb9c99aae38df53d9"
format_stamp: "Formatted at 2026-09-28 23:07:15 on dist-test-slave-x2t8"
server_key: "699e7d097629a11ed7862ab21880dd01"
server_key_iv: "e10d03e38380302f05a3db4d4458b024"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.726274 24856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.747720 24856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.749444 24856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.749547 24856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.753135 24856 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46143
I20260928 23:07:16.753163 24923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46143 every 8 connection(s)
I20260928 23:07:16.753527 24856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:16.754858 24924 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:07:16.755465 24924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Bootstrap starting.
I20260928 23:07:16.756794 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24856
I20260928 23:07:16.757082 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:37935
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:46143
--webserver_port=37327
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:07:16.763737 24924 log.cc:824] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:16.765324 24924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:07:16.765597 24924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Bootstrap complete.
I20260928 23:07:16.767715 24924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.767947 24924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d3373d360e4d9cb9c99aae38df53d9, State: Initialized, Role: FOLLOWER
I20260928 23:07:16.768146 24924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.768239 24924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:16.768290 24924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:16.768337 24924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:07:16.769289 24924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.769404 24924 leader_election.cc:302] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68d3373d360e4d9cb9c99aae38df53d9; no voters:
I20260928 23:07:16.769555 24924 leader_election.cc:288] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 23:07:16.769587 24928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 23:07:16.769704 24928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [term 2 LEADER]: Becoming Leader. State: Replica: 68d3373d360e4d9cb9c99aae38df53d9, State: Running, Role: LEADER
I20260928 23:07:16.769784 24928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } }
I20260928 23:07:16.769810 24924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:16.770119 24929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68d3373d360e4d9cb9c99aae38df53d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } } }
I20260928 23:07:16.770198 24929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:16.770421 24932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:16.770913 24932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:16.771682 24930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68d3373d360e4d9cb9c99aae38df53d9. Latest consensus state: current_term: 2 leader_uuid: "68d3373d360e4d9cb9c99aae38df53d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68d3373d360e4d9cb9c99aae38df53d9" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46143 } } }
I20260928 23:07:16.771755 24930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:16.771806 24932 catalog_manager.cc:1259] Loaded cluster ID: 776b62de77934a479242058d835ba614
I20260928 23:07:16.771832 24932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:16.772995 24932 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:16.773154 24932 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68d3373d360e4d9cb9c99aae38df53d9: Loaded TSK: 0
I20260928 23:07:16.773357 24932 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 23:07:16.837370 24926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.837610 24926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.837647 24926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.838802 24926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.838878 24926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.838922 24926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:16.840327 24926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.65.keytab
--principal=oryx/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:37935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=37327
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.840602 24926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.840833 24926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:16.846716 24951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:16.846716 24950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.846911 24926 server_base.cc:1034] running on GCE node
W20260928 23:07:16.846722 24953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:16.847183 24926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:16.847404 24926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:16.848546 24926 hybrid_clock.cc:648] HybridClock initialized: now 1790636836848534 us; error 30 us; skew 500 ppm
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636836, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:16.851166 24926 init.cc:377] Logged in from keytab as oryx/127.15.212.65@KRBTEST.COM (short username oryx)
I20260928 23:07:16.851701 24926 webserver.cc:460] Webserver started at http://127.15.212.65:37327/ using document root <none> and password file <none>
I20260928 23:07:16.851961 24926 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:16.852044 24926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:16.853899 24926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.854557 24960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.854765 24926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:16.854858 24926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "cc12304e4c8d4444b3bc7dbb2e71c771"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "8598650b355f58580bad91b74fa0390e"
server_key_iv: "20624ae2b29526bf2445f8d95c9fed8f"
server_key_version: "encryptionkey@0"
I20260928 23:07:16.855280 24926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:16.898187 24926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:16.900539 24926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:16.900650 24926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:16.900835 24926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:16.901108 24926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:16.901141 24926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.901163 24926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:16.901175 24926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:16.906826 24926 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:37935
I20260928 23:07:16.906872 25073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:37935 every 8 connection(s)
I20260928 23:07:16.907209 24926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:07:16.912387 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 24926
I20260928 23:07:16.912747 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:38089
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:46143
--webserver_port=41675
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 23:07:16.915922 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:16.908008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:34801 (local address 127.15.212.126:46143)
0928 23:07:16.908179 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.908187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.908246 (+ 59us) server_negotiation.cc:407] Connection header received
0928 23:07:16.908935 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.908941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.908993 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.909150 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.910081 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.910949 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.911937 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.912485 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.912596 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.915805 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:34801: BlockingRecv error: recv got EOF from 127.15.212.65:34801 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260928 23:07:16.915918 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.907930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:34801)
0928 23:07:16.908085 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.908152 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.908737 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.909190 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.909194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.909768 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.909951 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.909955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.911054 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.911055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.911843 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.911845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.912600 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.912601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.912738 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.913335 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.913351 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.915702 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
W20260928 23:07:16.916193 25074 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:46143 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.212.126:46143: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:16.921119 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.916460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:55631)
0928 23:07:16.916509 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.916533 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.916603 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.916736 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.916738 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.916896 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.916953 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.916955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.917598 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.917599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.918523 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.918526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.918868 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.918870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.918946 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.919317 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.919322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.921077 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 23:07:16.921259 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:16.916494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:55631 (local address 127.15.212.126:46143)
0928 23:07:16.916532 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.916534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.916592 (+ 58us) server_negotiation.cc:407] Connection header received
0928 23:07:16.916665 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.916666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.916677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.916731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.917040 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.917521 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.918615 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.918792 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.918849 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.921183 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:55631: BlockingRecv error: recv got EOF from 127.15.212.65:55631 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:16 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:16.925441 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:16.921522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:56625)
0928 23:07:16.921565 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:16.921585 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:16.921637 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:16.921775 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:16.921775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:16.921896 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:16.921918 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.921919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.922494 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.922495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.922987 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:16.922988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.923384 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.923385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:16.923438 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.923835 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:16.923840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:16.925367 (+ 1527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:07:16.925484 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:16.921589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:56625 (local address 127.15.212.126:46143)
0928 23:07:16.921645 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:16.921647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:16.921656 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:16.921682 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:16.921683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:16.921693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:16.921769 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:16.921987 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.922415 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.923086 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:16.923295 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:16.923384 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:16.925428 (+ 2044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:56625: BlockingRecv error: recv got EOF from 127.15.212.65:56625 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 23:07:16.925544 25074 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:07:16.993189 25078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:16.993426 25078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:16.993463 25078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:16.994738 25078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:16.994824 25078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:16.994871 25078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:16.996536 25078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.66.keytab
--principal=oryx/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:38089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=41675
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:16.996799 25078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:16.997017 25078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:17.002161 25083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:17.002286 25078 server_base.cc:1034] running on GCE node
W20260928 23:07:17.002116 25084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:17.002116 25086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:17.002599 25078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:17.002797 25078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:17.003960 25078 hybrid_clock.cc:648] HybridClock initialized: now 1790636837003957 us; error 35 us; skew 500 ppm
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636837, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:17.006345 25078 init.cc:377] Logged in from keytab as oryx/127.15.212.66@KRBTEST.COM (short username oryx)
I20260928 23:07:17.006834 25078 webserver.cc:460] Webserver started at http://127.15.212.66:41675/ using document root <none> and password file <none>
I20260928 23:07:17.007063 25078 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:17.007146 25078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:17.008424 25078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:17.009002 25093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.009192 25078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.009285 25078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "28cdcb9f036c4beab840c531d0951a05"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "4a91fc88e396fdadae93a517ac6eca05"
server_key_iv: "34c5c72d8df3e2c0476a50d2aeb77a4d"
server_key_version: "encryptionkey@0"
I20260928 23:07:17.009619 25078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:17.020438 25078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:17.022293 25078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:17.022447 25078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:17.022663 25078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:17.022939 25078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:17.022990 25078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.023021 25078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:17.023046 25078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.028862 25078 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:38089
I20260928 23:07:17.028911 25206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:38089 every 8 connection(s)
I20260928 23:07:17.029217 25078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 23:07:17.036239 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.029868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:58645 (local address 127.15.212.126:46143)
0928 23:07:17.029935 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.029937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.030150 (+ 213us) server_negotiation.cc:407] Connection header received
0928 23:07:17.030586 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.030587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.030600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.030650 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.031491 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.031966 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.033005 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.033206 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.033262 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.036180 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:58645: BlockingRecv error: recv got EOF from 127.15.212.66:58645 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:07:17.036307 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.029899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:58645)
0928 23:07:17.030039 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.030071 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.030493 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.030698 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.030701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.031237 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.031411 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.031416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.032048 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.032048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.032935 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.032937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.033278 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.033278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.033370 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.033756 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.033764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.036115 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
W20260928 23:07:17.036535 25207 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:46143 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.212.126:46143: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
I20260928 23:07:17.038161 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25078
I20260928 23:07:17.038465 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:33479
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:46143
--webserver_port=44875
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:17.041014 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.036847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:36447)
0928 23:07:17.036900 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.036912 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.036956 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.037065 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.037065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.037169 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.037211 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.037212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.037865 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.037866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.038395 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.038397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.038832 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.038833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.038897 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.039313 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.039318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.040939 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 23:07:17.041091 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.036918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:36447 (local address 127.15.212.126:46143)
0928 23:07:17.036969 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.036971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.036980 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:17.037001 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.037002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.037010 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.037060 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.037288 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.037788 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.038475 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.038759 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.038834 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.041016 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:36447: BlockingRecv error: recv got EOF from 127.15.212.66:36447 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:17.045878 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.041449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:44237)
0928 23:07:17.041493 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.041503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.041535 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.041695 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.041696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.041854 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.041890 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.041891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.042483 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.042484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.043377 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.043379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.043708 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.043709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.043771 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.044094 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.044099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.045806 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 23:07:17.045928 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.041495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:44237 (local address 127.15.212.126:46143)
0928 23:07:17.041528 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.041533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.041540 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:17.041591 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.041592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.041598 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.041675 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.041971 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.042409 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.043459 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.043638 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.043684 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.045870 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:44237: BlockingRecv error: recv got EOF from 127.15.212.66:44237 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 23:07:17.046017 25207 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:07:17.127228 25210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:17.127461 25210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:17.127511 25210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:17.128799 25210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:17.128875 25210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:17.128922 25210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:17.130475 25210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/krb5kdc/oryx_127.15.212.67.keytab
--principal=oryx/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:33479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=44875
--tserver_master_addrs=127.15.212.126:46143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:17.130760 25210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:17.131004 25210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:17.135857 25215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:17.135864 25216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:17.135893 25218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:17.135921 25210 server_base.cc:1034] running on GCE node
I20260928 23:07:17.136264 25210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:17.136528 25210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:17.137691 25210 hybrid_clock.cc:648] HybridClock initialized: now 1790636837137684 us; error 41 us; skew 500 ppm
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636837, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:17.140025 25210 init.cc:377] Logged in from keytab as oryx/127.15.212.67@KRBTEST.COM (short username oryx)
I20260928 23:07:17.140422 25210 webserver.cc:460] Webserver started at http://127.15.212.67:44875/ using document root <none> and password file <none>
I20260928 23:07:17.140728 25210 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:17.140805 25210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:17.142089 25210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:07:17.142685 25225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.142864 25210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:17.142952 25210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "4415cbedb9bf4a2d8d9df11c8a72feb8"
format_stamp: "Formatted at 2026-09-28 23:07:16 on dist-test-slave-x2t8"
server_key: "461b97fb8fdc20b9fd6561df353b8525"
server_key_iv: "92b6bf798632eb3a850dbf5137503fd3"
server_key_version: "encryptionkey@0"
I20260928 23:07:17.143281 25210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:17.164227 25210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:17.165920 25210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:17.166074 25210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:17.166290 25210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:17.166568 25210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:17.166623 25210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.166663 25210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:17.166692 25210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:17.172467 25210 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:33479
I20260928 23:07:17.172521 25338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:33479 every 8 connection(s)
I20260928 23:07:17.172858 25210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:17.173452 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25210
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 23:07:17.181792 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.173576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57385 (local address 127.15.212.126:46143)
0928 23:07:17.173627 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.173629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.173885 (+ 256us) server_negotiation.cc:407] Connection header received
0928 23:07:17.174540 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.174541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.174552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.174618 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.175767 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.176488 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.177910 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.178238 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.178317 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.181718 (+ 3401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:57385: BlockingRecv error: recv got EOF from 127.15.212.67:57385 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:07:17.181836 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.173553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:57385)
0928 23:07:17.173734 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.173780 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.174438 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.174661 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.174664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.175371 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.175660 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.175666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.176561 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.176562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.177824 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.177827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.178329 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.178330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.178437 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.178970 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.179005 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.181633 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260928 23:07:17.181886 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.173966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43890 (local address 127.15.212.126:46143)
0928 23:07:17.174089 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.174092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.174167 (+ 75us) server_negotiation.cc:407] Connection header received
0928 23:07:17.174225 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.174226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.174240 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.174316 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.174968 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.175818 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.176846 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.177184 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.177259 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.178593 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:17.178597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:17.178598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:17.178610 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:17.180391 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:17.180779 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:17.180783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:17.180783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:17.180910 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:17.181107 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:17.181108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:17.181108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:17.181351 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:17.181455 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:17.181638 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 23:07:17.181756 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
W20260928 23:07:17.182065 25339 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:46143 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.212.126:46143: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:17.186224 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.182290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:39443)
0928 23:07:17.182336 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.182346 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.182381 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.182527 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.182528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.182681 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.182733 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.182735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.183290 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.183291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.183839 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.183841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.184159 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.184160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.184212 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.184601 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.184605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.186184 (+ 1579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:07:17.186321 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.182332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:39443 (local address 127.15.212.126:46143)
0928 23:07:17.182365 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.182367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.182376 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:17.182433 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.182434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.182442 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.182503 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.182798 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.183222 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.183922 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.184085 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.184131 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.186258 (+ 2127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:39443: BlockingRecv error: recv got EOF from 127.15.212.67:39443 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:17.190007 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.186539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:43269)
0928 23:07:17.186594 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.186605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.186649 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.186751 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.186751 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.186850 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.186873 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.186874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.187389 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.187390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.187838 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.187839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.188139 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.188140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.188183 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.188546 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.188551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.189969 (+ 1418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 23:07:17.190109 25077 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.186606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:43269 (local address 127.15.212.126:46143)
0928 23:07:17.186657 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.186659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.186666 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:17.186698 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.186699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.186705 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.186747 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.186923 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.187330 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.187900 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.188060 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.188106 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.190042 (+ 1936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:43269: BlockingRecv error: recv got EOF from 127.15.212.67:43269 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 23:07:17.190164 25339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:17 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:17.930351 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:17.925896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:48781)
0928 23:07:17.926022 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:17.926034 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:17.926091 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:17.926255 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:17.926256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:17.926412 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:17.926460 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.926462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.927076 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.927077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.927621 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:17.927623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.927936 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.927937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:17.927975 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.928472 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:17.928476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:17.930284 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
W20260928 23:07:17.930459 25345 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:17.925946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48781 (local address 127.15.212.126:46143)
0928 23:07:17.926073 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:17.926075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:17.926092 (+ 17us) server_negotiation.cc:407] Connection header received
0928 23:07:17.926137 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:17.926138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:17.926147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:17.926241 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:17.926523 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.927013 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.927688 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:17.927874 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:17.927924 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:17.930381 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:48781: BlockingRecv error: recv got EOF from 127.15.212.65:48781 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:18.050951 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:18.046325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:34901)
0928 23:07:18.046438 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:18.046448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:18.046513 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:18.046709 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:18.046710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:18.046935 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:18.046992 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.046994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.047648 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.047649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.048206 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.048207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.048617 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.048618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.048677 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.049146 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:18.049150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:18.050884 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
W20260928 23:07:18.051044 25345 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:18.046390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34901 (local address 127.15.212.126:46143)
0928 23:07:18.046453 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:18.046455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:18.046499 (+ 44us) server_negotiation.cc:407] Connection header received
0928 23:07:18.046593 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:18.046594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:18.046603 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:18.046699 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:18.047074 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.047558 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.048289 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.048529 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.048619 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.050996 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:34901: BlockingRecv error: recv got EOF from 127.15.212.66:34901 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:18.195441 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:18.190609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:58103)
0928 23:07:18.190795 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:18.190809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:18.190908 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:18.191094 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:18.191094 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:18.191289 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:18.191334 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.191336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.192014 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.192015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.192606 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.192608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.192985 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.192986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.193052 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.193575 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:18.193580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:18.195334 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
W20260928 23:07:18.195482 25345 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:18.190714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:58103 (local address 127.15.212.126:46143)
0928 23:07:18.190799 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:18.190801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:18.190888 (+ 87us) server_negotiation.cc:407] Connection header received
0928 23:07:18.190988 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:18.190989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:18.191004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:18.191087 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:18.191421 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.191923 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.192701 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.192898 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.192984 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.195418 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:58103: BlockingRecv error: recv got EOF from 127.15.212.67:58103 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:18 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:18.935379 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:18.930830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:53499)
0928 23:07:18.930970 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:18.930982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:18.931061 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:18.931203 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:18.931204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:18.931362 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:18.931412 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.931413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.932048 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.932049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.932585 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:18.932587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.932984 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.932986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:18.933048 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.933526 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:18.933530 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:18.935283 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
W20260928 23:07:18.935432 25349 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:18.930920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:53499 (local address 127.15.212.126:46143)
0928 23:07:18.931055 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:18.931057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:18.931065 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:18.931127 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:18.931127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:18.931137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:18.931189 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:18.931476 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.931982 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.932676 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:18.932917 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:18.933001 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:18.935361 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:53499: BlockingRecv error: recv got EOF from 127.15.212.65:53499 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:19.056234 25350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:19.051523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:49417)
0928 23:07:19.051646 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:19.051657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:19.051724 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:19.051893 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:19.051894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:19.052101 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:19.052155 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.052156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.052884 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.052885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.053418 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.053420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.053786 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.053787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.053835 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.054333 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:19.054339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:19.056140 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260928 23:07:19.056277 25349 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:19.051589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:49417 (local address 127.15.212.126:46143)
0928 23:07:19.051647 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:19.051649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:19.051725 (+ 76us) server_negotiation.cc:407] Connection header received
0928 23:07:19.051798 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:19.051798 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:19.051812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:19.051872 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:19.052218 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.052792 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.053497 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.053698 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.053755 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.056217 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:49417: BlockingRecv error: recv got EOF from 127.15.212.66:49417 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:19.200579 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:19.195972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:37083)
0928 23:07:19.196109 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:19.196120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:19.196183 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:19.196310 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:19.196311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:19.196473 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:19.196519 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.196521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.197190 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.197191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.197739 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.197741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.198108 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.198109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.198165 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.198666 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:19.198671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:19.200485 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260928 23:07:19.200598 25349 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:19.196132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:37083 (local address 127.15.212.126:46143)
0928 23:07:19.196186 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:19.196188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:19.196223 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:07:19.196242 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:19.196242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:19.196253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:19.196290 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:19.196627 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.197112 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.197825 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.198017 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.198108 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.200544 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:37083: BlockingRecv error: recv got EOF from 127.15.212.67:37083 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:19 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:19.940099 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:19.935826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:51335)
0928 23:07:19.935952 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:19.935963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:19.936023 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:19.936157 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:19.936157 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:19.936309 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:19.936356 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.936358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.937000 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.937000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.937567 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:19.937569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.937896 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.937896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:19.937933 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.938352 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:19.938356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:19.940040 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
W20260928 23:07:19.940207 25353 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:19.935904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:51335 (local address 127.15.212.126:46143)
0928 23:07:19.936040 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:19.936041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:19.936049 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:19.936069 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:19.936069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:19.936080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:19.936142 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:19.936416 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.936937 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.937640 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:19.937831 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:19.937885 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:19.940127 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:51335: BlockingRecv error: recv got EOF from 127.15.212.65:51335 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:20.062609 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:20.056755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:47283)
0928 23:07:20.056962 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:20.057009 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:20.057130 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:20.057335 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:20.057336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:20.057517 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:20.057563 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.057565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.058309 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.058310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.059226 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.059229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.059705 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.059706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.059757 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.060261 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:20.060267 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:20.062550 (+ 2283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260928 23:07:20.062927 25353 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:20.056804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:47283 (local address 127.15.212.126:46143)
0928 23:07:20.056855 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:20.056858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:20.057109 (+ 251us) server_negotiation.cc:407] Connection header received
0928 23:07:20.057193 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:20.057193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:20.057238 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:20.057302 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:20.057649 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.058225 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.059325 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.059644 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.059709 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.062846 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:47283: BlockingRecv error: recv got EOF from 127.15.212.66:47283 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:20.205623 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:20.201237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:53809)
0928 23:07:20.201385 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:20.201398 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:20.201463 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:20.201627 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:20.201627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:20.201778 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:20.201826 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.201828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.202437 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.202438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.202982 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.202984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.203398 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.203399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.203441 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.203850 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:20.203854 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:20.205566 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
W20260928 23:07:20.205694 25353 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:20.201306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:53809 (local address 127.15.212.126:46143)
0928 23:07:20.201390 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:20.201392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:20.201468 (+ 76us) server_negotiation.cc:407] Connection header received
0928 23:07:20.201525 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:20.201526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:20.201546 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:20.201604 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:20.201894 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.202375 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.203055 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.203331 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.203386 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.205650 (+ 2264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:53809: BlockingRecv error: recv got EOF from 127.15.212.67:53809 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:20 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:20.945029 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:20.940585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:59005)
0928 23:07:20.940741 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:20.940752 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:20.940816 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:20.940930 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:20.940931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:20.941085 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:20.941132 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.941134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.941812 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.941813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.942352 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:20.942355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.942731 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.942732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:20.942776 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.943218 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:20.943224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:20.944973 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
W20260928 23:07:20.945163 25357 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:20.940691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:59005 (local address 127.15.212.126:46143)
0928 23:07:20.940815 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:20.940817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:20.940825 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:20.940861 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:20.940862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:20.940871 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:20.940925 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:20.941188 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.941715 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.942429 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:20.942650 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:20.942712 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:20.945061 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:59005: BlockingRecv error: recv got EOF from 127.15.212.65:59005 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":47,"threads_started":1}
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:21.071234 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:21.063166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:35641)
0928 23:07:21.063325 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:21.063341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:21.063403 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:21.063515 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:21.063516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:21.063675 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:21.063721 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.063723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.064333 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.064333 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.064872 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.064873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.065187 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.065188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.065225 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.069116 (+ 3891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:21.069122 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:21.071133 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
W20260928 23:07:21.071283 25357 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:21.063271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:35641 (local address 127.15.212.126:46143)
0928 23:07:21.063341 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:21.063343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:21.063405 (+ 62us) server_negotiation.cc:407] Connection header received
0928 23:07:21.063450 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:21.063450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:21.063461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:21.063508 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:21.063782 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.064262 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.064944 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.065127 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.065179 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.071219 (+ 6040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:35641: BlockingRecv error: recv got EOF from 127.15.212.66:35641 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:21.210740 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:21.206139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:40341)
0928 23:07:21.206306 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:21.206317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:21.206383 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:21.206510 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:21.206511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:21.206670 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:21.206718 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.206720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.207349 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.207350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.207873 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.207874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.208203 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.208204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.208261 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.208812 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:21.208816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:21.210684 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
W20260928 23:07:21.210944 25357 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:21.206259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:40341 (local address 127.15.212.126:46143)
0928 23:07:21.206311 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:21.206313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:21.206368 (+ 55us) server_negotiation.cc:407] Connection header received
0928 23:07:21.206437 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:21.206438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:21.206449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:21.206493 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:21.206786 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.207280 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.207949 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.208134 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.208187 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.210769 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:40341: BlockingRecv error: recv got EOF from 127.15.212.67:40341 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:21 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:21.950089 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:21.945538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:48109)
0928 23:07:21.945687 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:21.945700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:21.945813 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:21.946001 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:21.946002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:21.946185 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:21.946236 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.946237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.946877 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.946878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.947415 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:21.947417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.947757 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.947758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:21.947798 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.948216 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:21.948221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:21.950034 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":103,"threads_started":1}
W20260928 23:07:21.950241 25361 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:21.945647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48109 (local address 127.15.212.126:46143)
0928 23:07:21.945796 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:21.945798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:21.945807 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:21.945881 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:21.945882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:21.945908 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:21.945989 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:21.946303 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.946802 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.947494 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:21.947689 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:21.947742 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:21.950148 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:48109: BlockingRecv error: recv got EOF from 127.15.212.65:48109 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:22.076756 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:22.071691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:42891)
0928 23:07:22.071820 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:22.071830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:22.071907 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:22.072058 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:22.072059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:22.072235 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:22.072282 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.072283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.072997 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.072998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.073801 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.073803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.074203 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.074204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.074260 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.074750 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:22.074756 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:22.076684 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
W20260928 23:07:22.076856 25361 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:22.071824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:42891 (local address 127.15.212.126:46143)
0928 23:07:22.071890 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:22.071892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:22.071902 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:22.071965 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:22.071966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:22.071978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:22.072050 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:22.072366 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.072912 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.073899 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.074096 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.074207 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.076778 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:42891: BlockingRecv error: recv got EOF from 127.15.212.66:42891 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:22.215924 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:22.211167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:38181)
0928 23:07:22.211302 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:22.211313 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:22.211399 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:22.211532 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:22.211533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:22.211695 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:22.211742 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.211743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.212502 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.212503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.213227 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.213229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.213676 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.213677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.213736 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.214134 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:22.214138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:22.215833 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
W20260928 23:07:22.215960 25361 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:22.211229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:38181 (local address 127.15.212.126:46143)
0928 23:07:22.211307 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:22.211310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:22.211392 (+ 82us) server_negotiation.cc:407] Connection header received
0928 23:07:22.211441 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:22.211441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:22.211453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:22.211521 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:22.211802 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.212416 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.213330 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.213597 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.213677 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.215901 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:38181: BlockingRecv error: recv got EOF from 127.15.212.67:38181 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:22 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:22.955317 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:22.950513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:44523)
0928 23:07:22.950656 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:22.950666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:22.950713 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:22.950842 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:22.950843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:22.950999 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:22.951047 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.951049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.951692 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.951693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.952212 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:22.952213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.952606 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.952607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:22.952669 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.953188 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:22.953192 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:22.955221 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
W20260928 23:07:22.955370 25365 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:22.950594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44523 (local address 127.15.212.126:46143)
0928 23:07:22.950723 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:22.950725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:22.950732 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:07:22.950753 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:22.950753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:22.950763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:22.950823 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:22.951113 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.951625 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.952282 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:22.952514 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:22.952567 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:22.955300 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:44523: BlockingRecv error: recv got EOF from 127.15.212.65:44523 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:23.081694 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:23.077344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:41199)
0928 23:07:23.077497 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:23.077528 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:23.077588 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:23.077707 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:23.077708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:23.077861 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:23.077908 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.077909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.078514 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.078515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.079028 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.079030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.079347 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.079347 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.079383 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.079866 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:23.079871 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:23.081638 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
W20260928 23:07:23.081848 25365 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:23.077401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:41199 (local address 127.15.212.126:46143)
0928 23:07:23.077520 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:23.077522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:23.077573 (+ 51us) server_negotiation.cc:407] Connection header received
0928 23:07:23.077641 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:23.077642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:23.077652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:23.077691 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:23.077970 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.078450 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.079105 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.079286 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.079338 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.081752 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:41199: BlockingRecv error: recv got EOF from 127.15.212.66:41199 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:23.220731 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:23.216345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:55379)
0928 23:07:23.216494 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:23.216517 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:23.216578 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:23.216717 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:23.216718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:23.216868 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:23.216915 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.216917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.217550 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.217551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.218059 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.218060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.218373 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.218374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.218410 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.218841 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:23.218845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:23.220673 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
W20260928 23:07:23.220821 25365 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:23.216465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:55379 (local address 127.15.212.126:46143)
0928 23:07:23.216523 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:23.216525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:23.216563 (+ 38us) server_negotiation.cc:407] Connection header received
0928 23:07:23.216636 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:23.216637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:23.216647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:23.216699 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:23.216974 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.217440 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.218127 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.218310 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.218365 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.220759 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:55379: BlockingRecv error: recv got EOF from 127.15.212.67:55379 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:23 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:23.960512 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:23.955830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:52537)
0928 23:07:23.955967 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:23.955978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:23.956039 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:23.956186 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:23.956187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:23.956339 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:23.956389 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.956391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.957053 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.957054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.957587 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:23.957589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.957908 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.957908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:23.957946 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.958452 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:23.958457 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:23.960404 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260928 23:07:23.960623 25369 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:23.955892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:52537 (local address 127.15.212.126:46143)
0928 23:07:23.956022 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:23.956024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:23.956033 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:23.956097 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:23.956098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:23.956107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:23.956160 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:23.956474 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.956979 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.957661 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:23.957845 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:23.957894 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:23.960528 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:52537: BlockingRecv error: recv got EOF from 127.15.212.65:52537 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:24.087028 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:24.082213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:60997)
0928 23:07:24.082352 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:24.082362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:24.082417 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:24.082544 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:24.082544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:24.082697 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:24.082745 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.082746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.083414 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.083415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.083953 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.083955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.084255 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.084256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.084291 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.084976 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:24.084982 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:24.086962 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
W20260928 23:07:24.087081 25369 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:24.082274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:60997 (local address 127.15.212.126:46143)
0928 23:07:24.082341 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:24.082343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:24.082421 (+ 78us) server_negotiation.cc:407] Connection header received
0928 23:07:24.082470 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:24.082471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:24.082482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:24.082523 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:24.082805 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.083350 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.084022 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.084197 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.084261 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.087036 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:60997: BlockingRecv error: recv got EOF from 127.15.212.66:60997 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:24.225919 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:24.221191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:55391)
0928 23:07:24.221357 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:24.221369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:24.221454 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:24.221654 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:24.221655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:24.221832 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:24.221876 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.221877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.222548 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.222549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.223098 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.223100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.223475 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.223476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.223534 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.223992 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:24.223996 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:24.225823 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
W20260928 23:07:24.225970 25369 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:24.221281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:55391 (local address 127.15.212.126:46143)
0928 23:07:24.221345 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:24.221347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:24.221435 (+ 88us) server_negotiation.cc:407] Connection header received
0928 23:07:24.221554 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:24.221555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:24.221568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:24.221636 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:24.221962 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.222456 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.223188 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.223399 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.223477 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.225905 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:55391: BlockingRecv error: recv got EOF from 127.15.212.67:55391 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:24 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:24.965231 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:24.960901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:48451)
0928 23:07:24.961056 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:24.961066 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:24.961114 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:24.961249 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:24.961250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:24.961399 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:24.961448 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.961449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.962139 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.962140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.962650 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:24.962651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.962968 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.962968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:24.963004 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.963419 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:24.963423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:24.965176 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260928 23:07:24.965372 25373 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:24.961016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48451 (local address 127.15.212.126:46143)
0928 23:07:24.961138 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:24.961140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:24.961148 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:24.961167 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:24.961168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:24.961177 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:24.961233 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:24.961549 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.962055 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.962721 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:24.962907 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:24.962957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:24.965281 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:48451: BlockingRecv error: recv got EOF from 127.15.212.65:48451 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:25.092077 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:25.087637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:56121)
0928 23:07:25.087766 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:25.087775 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:25.087827 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:25.087967 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:25.087968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:25.088114 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:25.088180 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.088182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.088976 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.088977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.089487 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.089489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.089801 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.089802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.089837 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.090263 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:25.090267 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:25.092010 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260928 23:07:25.092187 25373 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:25.087673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:56121 (local address 127.15.212.126:46143)
0928 23:07:25.087741 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:25.087743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:25.087829 (+ 86us) server_negotiation.cc:407] Connection header received
0928 23:07:25.087878 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:25.087879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:25.087891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:25.087948 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:25.088244 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.088917 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.089560 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.089739 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.089791 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.092094 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:56121: BlockingRecv error: recv got EOF from 127.15.212.66:56121 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:25.231087 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:25.226452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:38253)
0928 23:07:25.226628 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:25.226639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:25.226732 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:25.226885 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:25.226886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:25.227084 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:25.227128 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.227130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.227807 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.227808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.228341 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.228342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.228754 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.228755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.228833 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.229302 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:25.229306 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:25.230981 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
W20260928 23:07:25.231118 25373 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:25.226531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:38253 (local address 127.15.212.126:46143)
0928 23:07:25.226620 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:25.226622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:25.226715 (+ 93us) server_negotiation.cc:407] Connection header received
0928 23:07:25.226819 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:25.226820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:25.226833 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:25.226881 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:25.227214 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.227721 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.228430 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.228665 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.228752 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.231055 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:38253: BlockingRecv error: recv got EOF from 127.15.212.67:38253 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:25 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:25.970177 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:25.965669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:44253)
0928 23:07:25.965813 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:25.965823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:25.965869 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:25.966020 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:25.966021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:25.966174 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:25.966221 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.966223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.966831 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.966831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.967343 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:25.967344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.967706 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.967707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:25.967755 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.968173 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:25.968177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:25.970051 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
W20260928 23:07:25.970227 25377 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:25.965726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44253 (local address 127.15.212.126:46143)
0928 23:07:25.965847 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:25.965849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:25.965862 (+ 13us) server_negotiation.cc:407] Connection header received
0928 23:07:25.965926 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:25.965927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:25.965936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:25.965994 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:25.966283 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.966767 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.967413 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:25.967604 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:25.967659 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:25.970125 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:44253: BlockingRecv error: recv got EOF from 127.15.212.65:44253 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:26.101890 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:26.092537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:33543)
0928 23:07:26.092712 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:26.092726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:26.092806 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:26.092970 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:26.092971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:26.093181 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:26.093241 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.093244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.093852 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.093852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.094358 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.094359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.094651 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.094651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.094688 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.100029 (+ 5341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:26.100033 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:26.101835 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
W20260928 23:07:26.101966 25377 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:26.092645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:33543 (local address 127.15.212.126:46143)
0928 23:07:26.092728 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:26.092730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:26.092789 (+ 59us) server_negotiation.cc:407] Connection header received
0928 23:07:26.092855 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:26.092856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:26.092868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:26.092954 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:26.093316 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.093780 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.094425 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.094593 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.094642 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.101922 (+ 7280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:33543: BlockingRecv error: recv got EOF from 127.15.212.66:33543 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:26.236182 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:26.231531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:46171)
0928 23:07:26.231730 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:26.231746 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:26.231840 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:26.231988 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:26.231989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:26.232183 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:26.232229 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.232231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.233044 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.233045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.233558 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.233559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.233916 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.233917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.233972 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.234367 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:26.234371 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:26.236127 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":104,"threads_started":1}
W20260928 23:07:26.236305 25377 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:26.231639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:46171 (local address 127.15.212.126:46143)
0928 23:07:26.231755 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:26.231757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:26.231818 (+ 61us) server_negotiation.cc:407] Connection header received
0928 23:07:26.231915 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:26.231916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:26.231931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:26.231982 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:26.232312 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.232965 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.233653 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.233847 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.233918 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.236228 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:46171: BlockingRecv error: recv got EOF from 127.15.212.67:46171 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:26 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:26.975332 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:26.970787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:60053)
0928 23:07:26.970910 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:26.970922 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:26.970990 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:26.971115 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:26.971117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:26.971271 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:26.971320 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.971322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.971934 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.971934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.972491 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:26.972495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.973031 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.973032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:26.973075 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.973575 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:26.973580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:26.975278 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":50,"threads_started":1}
W20260928 23:07:26.975466 25381 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:26.970846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:60053 (local address 127.15.212.126:46143)
0928 23:07:26.970988 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:26.970990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:26.970999 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:26.971032 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:26.971032 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:26.971042 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:26.971098 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:26.971384 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.971869 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.972577 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:26.972950 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:26.973024 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:26.975382 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:60053: BlockingRecv error: recv got EOF from 127.15.212.65:60053 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:27.106994 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:27.102392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:48881)
0928 23:07:27.102543 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:27.102555 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:27.102638 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:27.102805 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:27.102805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:27.102976 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:27.103023 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.103025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.103651 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.103651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.104202 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.104203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.104710 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.104710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.104759 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.105188 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:27.105192 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:27.106927 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
W20260928 23:07:27.107084 25381 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:27.102465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:48881 (local address 127.15.212.126:46143)
0928 23:07:27.102514 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:27.102516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:27.102622 (+ 106us) server_negotiation.cc:407] Connection header received
0928 23:07:27.102706 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:27.102707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:27.102719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:27.102777 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:27.103094 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.103575 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.104300 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.104628 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.104690 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.107017 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:48881: BlockingRecv error: recv got EOF from 127.15.212.66:48881 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:27.241191 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:27.236714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:54887)
0928 23:07:27.236890 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:27.236909 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:27.236972 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:27.237116 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:27.237117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:27.237271 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:27.237318 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.237320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.237946 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.237947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.238481 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.238482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.238790 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.238790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.238838 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.239335 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:27.239339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:27.241126 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
W20260928 23:07:27.241297 25381 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:27.236829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:54887 (local address 127.15.212.126:46143)
0928 23:07:27.236883 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:27.236885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:27.236974 (+ 89us) server_negotiation.cc:407] Connection header received
0928 23:07:27.237030 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:27.237031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:27.237041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:27.237092 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:27.237380 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.237873 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.238550 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.238728 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.238777 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.241206 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:54887: BlockingRecv error: recv got EOF from 127.15.212.67:54887 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:27 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:27.980691 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:27.975920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:44317)
0928 23:07:27.976060 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:27.976070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:27.976116 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:27.976258 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:27.976259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:27.976418 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:27.976486 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.976488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.977135 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.977136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.977778 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:27.977781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.978296 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.978298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:27.978357 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.978780 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:27.978785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:27.980578 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
W20260928 23:07:27.980724 25385 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:27.975981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:44317 (local address 127.15.212.126:46143)
0928 23:07:27.976125 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:27.976127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:27.976157 (+ 30us) server_negotiation.cc:407] Connection header received
0928 23:07:27.976177 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:27.976178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:27.976188 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:27.976237 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:27.976556 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.977061 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.977889 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:27.978231 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:27.978296 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:27.980654 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:44317: BlockingRecv error: recv got EOF from 127.15.212.65:44317 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:28.112167 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:28.107510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:44271)
0928 23:07:28.107661 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:28.107673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:28.107745 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:28.107957 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:28.107958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:28.108155 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:28.108200 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.108202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.108848 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.108849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.109413 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.109415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.109759 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.109760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.109797 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.110299 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:28.110304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:28.112103 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
W20260928 23:07:28.112335 25385 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:28.107497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:44271 (local address 127.15.212.126:46143)
0928 23:07:28.107584 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:28.107586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:28.107750 (+ 164us) server_negotiation.cc:407] Connection header received
0928 23:07:28.107816 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:28.107817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:28.107829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:28.107884 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:28.108275 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.108764 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.109507 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.109693 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.109751 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.112221 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:44271: BlockingRecv error: recv got EOF from 127.15.212.66:44271 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:28.246055 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:28.241674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:58609)
0928 23:07:28.241808 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:28.241820 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:28.241884 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:28.242011 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:28.242012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:28.242175 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:28.242217 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.242219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.242850 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.242850 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.243382 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.243383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.243701 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.243701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.243737 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.244152 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:28.244156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:28.245988 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
W20260928 23:07:28.246104 25385 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:28.241745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:58609 (local address 127.15.212.126:46143)
0928 23:07:28.241819 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:28.241821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:28.241897 (+ 76us) server_negotiation.cc:407] Connection header received
0928 23:07:28.241936 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:28.241937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:28.241948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:28.241998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:28.242276 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.242786 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.243455 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.243640 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.243693 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.246059 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:58609: BlockingRecv error: recv got EOF from 127.15.212.67:58609 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:28 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:28.985816 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:28.981309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:38659)
0928 23:07:28.981471 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:28.981481 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:28.981543 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:28.981743 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:28.981744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:28.981924 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:28.981973 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.981974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.982608 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.982609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.983194 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:28.983196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.983507 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.983508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:28.983547 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.983972 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:28.983983 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:28.985761 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
W20260928 23:07:28.985922 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:28.981373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:38659 (local address 127.15.212.126:46143)
0928 23:07:28.981538 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:28.981540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:28.981549 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:28.981620 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:28.981620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:28.981648 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:28.981720 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:28.982049 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.982537 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.983266 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:28.983447 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:28.983496 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:28.985850 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:38659: BlockingRecv error: recv got EOF from 127.15.212.65:38659 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":44,"threads_started":1}
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:29.117139 25390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:29.112592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:47671)
0928 23:07:29.112776 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:29.112791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:29.112870 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:29.113018 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:29.113019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:29.113203 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:29.113245 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.113247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.113908 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.113909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.114448 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.114450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.114815 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.114816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.114879 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.115292 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:29.115296 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:29.117072 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
W20260928 23:07:29.117244 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:29.112661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:47671 (local address 127.15.212.126:46143)
0928 23:07:29.112746 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:29.112749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:29.112856 (+ 107us) server_negotiation.cc:407] Connection header received
0928 23:07:29.112953 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:29.112954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:29.112965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:29.113014 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:29.113329 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.113826 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.114532 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.114732 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.114814 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.117172 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:47671: BlockingRecv error: recv got EOF from 127.15.212.66:47671 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:29.251204 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:29.246545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:51971)
0928 23:07:29.246699 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:29.246709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:29.246763 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:29.246910 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:29.246910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:29.247076 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:29.247120 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.247122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.247758 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.247759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.248275 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.248277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.248732 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.248733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.248813 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.249354 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:29.249359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:29.251143 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
W20260928 23:07:29.251362 25389 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:29.246600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:51971 (local address 127.15.212.126:46143)
0928 23:07:29.246655 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:29.246656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:29.246808 (+ 152us) server_negotiation.cc:407] Connection header received
0928 23:07:29.246836 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:29.246837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:29.246847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:29.246887 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:29.247181 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.247652 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.248355 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.248661 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.248730 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.251228 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:51971: BlockingRecv error: recv got EOF from 127.15.212.67:51971 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:29 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:29.995724 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:29.986299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:60841)
0928 23:07:29.986466 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:29.986481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:29.986563 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:29.986713 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:29.986714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:29.986878 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:29.986926 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.986928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.987574 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.987575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.988094 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:29.988096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.988536 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.988537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:29.988580 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.993844 (+ 5264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:29.993848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:29.995630 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
W20260928 23:07:29.995783 25393 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:29.986391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:60841 (local address 127.15.212.126:46143)
0928 23:07:29.986486 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:29.986488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:29.986547 (+ 59us) server_negotiation.cc:407] Connection header received
0928 23:07:29.986623 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:29.986624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:29.986635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:29.986693 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:29.986992 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.987489 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.988192 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:29.988403 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:29.988531 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:29.995712 (+ 7181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:60841: BlockingRecv error: recv got EOF from 127.15.212.65:60841 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":44,"threads_started":1}
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:30.123600 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:30.117655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:45399)
0928 23:07:30.117782 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:30.117793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:30.117843 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:30.118026 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:30.118027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:30.118232 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:30.118293 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.118296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.119071 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.119092 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.119865 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.119868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.120304 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.120305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.120369 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:30.121160 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:30.121168 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:30.123510 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
W20260928 23:07:30.123835 25393 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:30.117766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:45399 (local address 127.15.212.126:46143)
0928 23:07:30.117820 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:30.117822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:30.117831 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:30.117920 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:30.117921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:30.117933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:30.118018 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:30.118376 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.118993 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.119949 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.120212 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.120282 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:30.123606 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:45399: BlockingRecv error: recv got EOF from 127.15.212.66:45399 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:30.256022 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:30.251622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:45187)
0928 23:07:30.251771 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:30.251789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:30.251868 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:30.252039 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:30.252040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:30.252204 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:30.252248 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.252250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.252871 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.252872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.253392 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.253394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.253693 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.253693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.253732 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:30.254153 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:30.254157 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:30.255893 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
W20260928 23:07:30.256052 25393 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:30.251694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:45187 (local address 127.15.212.126:46143)
0928 23:07:30.251753 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:30.251755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:30.251868 (+ 113us) server_negotiation.cc:407] Connection header received
0928 23:07:30.251920 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:30.251920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:30.251955 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:30.252018 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:30.252315 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.252810 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.253460 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.253634 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.253681 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:30.255987 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:45187: BlockingRecv error: recv got EOF from 127.15.212.67:45187 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:30 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:31.000780 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:30.996208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:59455)
0928 23:07:30.996354 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:30.996366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:30.996444 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:30.996649 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:30.996650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:30.996810 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:30.996857 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.996859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.997498 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.997498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.998025 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:30.998026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.998358 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.998359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:30.998397 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:30.998858 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:30.998862 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:31.000678 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
W20260928 23:07:31.000821 25397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:30.996288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:59455 (local address 127.15.212.126:46143)
0928 23:07:30.996435 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:30.996437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:30.996446 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:30.996539 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:30.996540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:30.996552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:30.996620 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:30.996924 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.997415 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.998096 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:30.998291 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:30.998344 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:31.000752 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:59455: BlockingRecv error: recv got EOF from 127.15.212.65:59455 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
Sep 28 23:07:31 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:31 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:31.134291 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:31.124210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:34441)
0928 23:07:31.124461 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:31.124471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:31.124526 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:31.124667 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:31.124668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:31.124828 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:31.124874 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:31.124875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.125510 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.125511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:31.126026 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:31.126028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.126389 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.126389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:31.126428 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:31.132335 (+ 5907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:31.132339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:31.134196 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":175,"threads_started":1}
W20260928 23:07:31.134339 25397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:31.124333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34441 (local address 127.15.212.126:46143)
0928 23:07:31.124402 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:31.124404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:31.124551 (+ 147us) server_negotiation.cc:407] Connection header received
0928 23:07:31.124581 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:31.124582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:31.124594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:31.124649 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:31.124943 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.125436 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.126125 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.126321 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.126381 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:31.134277 (+ 7896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:34441: BlockingRecv error: recv got EOF from 127.15.212.66:34441 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 23:07:31 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:31 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.67@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:31.261397 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:31.256533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.67:59535)
0928 23:07:31.256714 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:31.256726 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:31.256793 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:31.256955 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:31.256956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:31.257133 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:31.257177 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:31.257179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.257868 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.257869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:31.258402 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:31.258404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.258770 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.258771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:31.258826 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:31.259329 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:31.259335 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:31.261340 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
W20260928 23:07:31.261518 25397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:31.256624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:59535 (local address 127.15.212.126:46143)
0928 23:07:31.256707 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:31.256709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:31.256773 (+ 64us) server_negotiation.cc:407] Connection header received
0928 23:07:31.256854 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:31.256854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:31.256867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:31.256936 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:31.257262 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.257785 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.258501 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:31.258698 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:31.258773 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:31.261421 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.67:59535: BlockingRecv error: recv got EOF from 127.15.212.67:59535 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 23:07:32 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:32 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.65@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:32.005776 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.001250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.65:59125)
0928 23:07:32.001378 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:32.001392 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:32.001444 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:32.001562 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:32.001562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:32.001719 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:32.001779 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.001780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.002432 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.002432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.002967 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.002969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.003294 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.003294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.003334 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.003851 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:32.003855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:32.005718 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
W20260928 23:07:32.005908 25401 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:32.001333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:59125 (local address 127.15.212.126:46143)
0928 23:07:32.001462 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.001464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.001472 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:07:32.001491 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.001491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.001501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.001556 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.001838 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.002361 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.003041 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.003232 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.003280 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.005799 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.65:59125: BlockingRecv error: recv got EOF from 127.15.212.65:59125 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
Sep 28 23:07:32 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 23:07:32 dist-test-slave-x2t8 krb5kdc[24343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.212.66@KRBTEST.COM for oryx/127.15.212.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 23:07:32.139698 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.134815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46143 (local address 127.15.212.66:40661)
0928 23:07:32.134974 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:32.134995 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:32.135081 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:32.135240 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:32.135241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:32.135390 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:32.135442 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.135443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.136173 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.136174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.136815 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.136817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.137124 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.137125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.137163 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.137640 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:32.137645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:32.139592 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.212.126:46143: Server oryx/127.15.212.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
W20260928 23:07:32.139735 25401 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:32.134889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:40661 (local address 127.15.212.126:46143)
0928 23:07:32.134978 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.134979 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.135064 (+ 85us) server_negotiation.cc:407] Connection header received
0928 23:07:32.135126 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.135127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.135137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.135187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.135507 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.136097 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.136883 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.137062 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.137115 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.139675 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.212.66:40661: BlockingRecv error: recv got EOF from 127.15.212.66:40661 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 23:07:32.181170 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24926
I20260928 23:07:32.185559 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25078
I20260928 23:07:32.189634 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25210
I20260928 23:07:32.193604 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 24856
2026-09-28T23:07:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17429 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T23:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:32Z Disabled control of system clock
I20260928 23:07:32.219776 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:43057
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:43057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 23:07:32.297500 25406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:32.297685 25406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:32.297704 25406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:32.298780 25406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:32.298813 25406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:32.298825 25406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:32.298836 25406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:32.298848 25406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:32.300022 25406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:43057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:43057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:32.300223 25406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:32.300429 25406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:07:32.305610 25406 server_base.cc:1034] running on GCE node
W20260928 23:07:32.305549 25411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.305528 25414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.305529 25412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.306093 25406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:32.306378 25406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:32.307515 25406 hybrid_clock.cc:648] HybridClock initialized: now 1790636852307502 us; error 31 us; skew 500 ppm
I20260928 23:07:32.308755 25406 webserver.cc:460] Webserver started at http://127.15.212.126:40239/ using document root <none> and password file <none>
I20260928 23:07:32.308977 25406 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:32.309028 25406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:32.309204 25406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:32.310213 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "87fa26fad6e0475ea789ed9986a7e759"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "09f92b188feb1645539367000416e0c9"
server_key_iv: "fba63200475ec74f46dd7665b0e61b13"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.310560 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "87fa26fad6e0475ea789ed9986a7e759"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "09f92b188feb1645539367000416e0c9"
server_key_iv: "fba63200475ec74f46dd7665b0e61b13"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.311848 25406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:32.312645 25420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.312793 25406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:32.312875 25406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "87fa26fad6e0475ea789ed9986a7e759"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "09f92b188feb1645539367000416e0c9"
server_key_iv: "fba63200475ec74f46dd7665b0e61b13"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.312925 25406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:32.325871 25406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:32.326097 25406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:32.326201 25406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:32.329823 25406 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:43057
I20260928 23:07:32.329876 25472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:43057 every 8 connection(s)
I20260928 23:07:32.330189 25406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:32.330852 25473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:32.333495 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25406
I20260928 23:07:32.333617 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:32.333813 16209 external_mini_cluster.cc:1428] Setting key 23d30132a5c13c6f79b94d2a2e3ccae3
I20260928 23:07:32.334066 25473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759: Bootstrap starting.
I20260928 23:07:32.334740 25473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:32.335031 25473 log.cc:824] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:32.335795 25473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759: No bootstrap required, opened a new log
I20260928 23:07:32.338145 25473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } }
I20260928 23:07:32.338322 25473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:32.338387 25473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fa26fad6e0475ea789ed9986a7e759, State: Initialized, Role: FOLLOWER
I20260928 23:07:32.338526 25473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } }
I20260928 23:07:32.338629 25473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:32.338692 25473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:32.338739 25473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:32.339366 25473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } }
I20260928 23:07:32.339501 25473 leader_election.cc:302] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87fa26fad6e0475ea789ed9986a7e759; no voters:
I20260928 23:07:32.339620 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.334317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.15.212.126:43057)
0928 23:07:32.334507 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.334514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.334538 (+ 24us) server_negotiation.cc:407] Connection header received
0928 23:07:32.335007 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.335014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.335078 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.335269 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.335468 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.336404 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.337618 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.337993 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.338078 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.338908 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:32.338911 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:32.338911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:32.339068 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:32.339098 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:32.339108 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:32.339169 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:32.339274 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 23:07:32.339330 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260928 23:07:32.339696 25473 leader_election.cc:288] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:32.339869 25479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:32.340036 25479 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [term 1 LEADER]: Becoming Leader. State: Replica: 87fa26fad6e0475ea789ed9986a7e759, State: Running, Role: LEADER
I20260928 23:07:32.340119 25479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } }
I20260928 23:07:32.340308 25473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:32.340493 25480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87fa26fad6e0475ea789ed9986a7e759. Latest consensus state: current_term: 1 leader_uuid: "87fa26fad6e0475ea789ed9986a7e759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } } }
I20260928 23:07:32.340602 25480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:32.340479 25478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87fa26fad6e0475ea789ed9986a7e759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fa26fad6e0475ea789ed9986a7e759" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 43057 } } }
I20260928 23:07:32.340726 25478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:32.341014 25482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:32.341446 25482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:32.342842 25482 catalog_manager.cc:1347] Generated new cluster ID: a39c83a474ae4237aafb545aef6efcf8
I20260928 23:07:32.342893 25482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:32.349027 25482 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:32.349661 25482 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:32.355087 25482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87fa26fad6e0475ea789ed9986a7e759: Generated new TSK 0
I20260928 23:07:32.355263 25482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:32.356572 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--builtin_ntp_servers=127.15.212.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 23:07:32.449527 25497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:32.449740 25497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:32.449762 25497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:32.450918 25497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:32.450959 25497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:32.450984 25497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:32.452332 25497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:32.452592 25497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:32.452919 25497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:32.458926 25503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.459195 25497 server_base.cc:1034] running on GCE node
W20260928 23:07:32.458926 25505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.458957 25502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.459539 25497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:32.459810 25497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:32.460973 25497 hybrid_clock.cc:648] HybridClock initialized: now 1790636852460960 us; error 32 us; skew 500 ppm
I20260928 23:07:32.462251 25497 webserver.cc:460] Webserver started at http://127.15.212.65:41755/ using document root <none> and password file <none>
I20260928 23:07:32.462509 25497 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:32.462579 25497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:32.462734 25497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:32.463783 25497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "5a65df85f09e4ffaae2d8a4c3ac61cbd"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "23dab4f9abb6a16a2fe91a89e2ce8125"
server_key_iv: "1863ecf3ed33504f92775501fedba2a8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.464148 25497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "5a65df85f09e4ffaae2d8a4c3ac61cbd"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "23dab4f9abb6a16a2fe91a89e2ce8125"
server_key_iv: "1863ecf3ed33504f92775501fedba2a8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.465591 25497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:32.466416 25511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.466591 25497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:32.466689 25497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "5a65df85f09e4ffaae2d8a4c3ac61cbd"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "23dab4f9abb6a16a2fe91a89e2ce8125"
server_key_iv: "1863ecf3ed33504f92775501fedba2a8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.466789 25497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:32.485807 25497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:32.486142 25497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:32.486294 25497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:32.486518 25497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:32.486864 25497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:32.486920 25497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.486963 25497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:32.486990 25497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.492547 25497 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:34061
I20260928 23:07:32.492595 25624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:34061 every 8 connection(s)
I20260928 23:07:32.492949 25497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:07:32.497967 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.493675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:46829 (local address 127.15.212.126:43057)
0928 23:07:32.493741 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.493748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.493955 (+ 207us) server_negotiation.cc:407] Connection header received
0928 23:07:32.494421 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.494422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.494430 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.494493 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.495197 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.495695 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.496709 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.496901 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.496963 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.497741 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:32.497742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:32.497742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:32.497746 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:32.497758 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:32.497762 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:32.497806 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:32.497887 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 23:07:32.497906 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:07:32.498142 25627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.493636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43057 (local address 127.15.212.65:46829)
0928 23:07:32.493824 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:32.493865 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:32.494338 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:32.494568 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:32.494571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:32.494579 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:32.495056 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.495061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.495778 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.495779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.496632 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.496634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.496979 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.496980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.497079 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.497607 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:32.497607 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:32.497641 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:32.497663 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:32.497818 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:32.497822 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:32.497824 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:07:32.497875 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:07:32.497948 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260928 23:07:32.498670 25625 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43057
I20260928 23:07:32.498759 25625 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:32.498900 25625 heartbeater.cc:511] Master 127.15.212.126:43057 requested a full tablet report, sending...
I20260928 23:07:32.499385 25437 ts_manager.cc:194] Registered new tserver with Master: 5a65df85f09e4ffaae2d8a4c3ac61cbd (127.15.212.65:34061)
I20260928 23:07:32.499984 25437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.65:46829
I20260928 23:07:32.502112 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25497
I20260928 23:07:32.502193 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:32.502345 16209 external_mini_cluster.cc:1428] Setting key 09f09ed3819c8b4005c330a3c8e4ab0f
I20260928 23:07:32.503067 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--builtin_ntp_servers=127.15.212.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 23:07:32.583220 25628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:32.583421 25628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:32.583442 25628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:32.584621 25628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:32.584664 25628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:32.584689 25628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:32.585947 25628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:32.586133 25628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:32.586411 25628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:32.591609 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.591634 25636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.591648 25633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.591923 25628 server_base.cc:1034] running on GCE node
I20260928 23:07:32.592095 25628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:32.592289 25628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:32.593422 25628 hybrid_clock.cc:648] HybridClock initialized: now 1790636852593401 us; error 37 us; skew 500 ppm
I20260928 23:07:32.594656 25628 webserver.cc:460] Webserver started at http://127.15.212.66:34943/ using document root <none> and password file <none>
I20260928 23:07:32.594906 25628 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:32.594954 25628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:32.595117 25628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:32.596374 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "aa002832c7034d769e246bb7ed6ce45d"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "2c2d8795d1d6ed1a155576563b62f2a6"
server_key_iv: "7499111a7df0ecea5fdaa94560283bf8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.596879 25628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "aa002832c7034d769e246bb7ed6ce45d"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "2c2d8795d1d6ed1a155576563b62f2a6"
server_key_iv: "7499111a7df0ecea5fdaa94560283bf8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.598301 25628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:32.599035 25642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.599232 25628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:32.599298 25628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "aa002832c7034d769e246bb7ed6ce45d"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "2c2d8795d1d6ed1a155576563b62f2a6"
server_key_iv: "7499111a7df0ecea5fdaa94560283bf8"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.599385 25628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:32.608286 25628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:32.608597 25628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:32.608749 25628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:32.608968 25628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:32.609277 25628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:32.609310 25628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.609361 25628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:32.609387 25628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.615032 25628 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:40385
I20260928 23:07:32.615155 25755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:40385 every 8 connection(s)
I20260928 23:07:32.615430 25628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:32.617442 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25628
I20260928 23:07:32.617535 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:32.617730 16209 external_mini_cluster.cc:1428] Setting key 0607adbffbfcc7303f7f5c7c1148d88c
I20260928 23:07:32.618742 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--builtin_ntp_servers=127.15.212.84:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 23:07:32.621344 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.616190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:47675 (local address 127.15.212.126:43057)
0928 23:07:32.616232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.616234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.616507 (+ 273us) server_negotiation.cc:407] Connection header received
0928 23:07:32.617056 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.617057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.617065 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.617123 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.617923 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.618617 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.619661 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.619963 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.620036 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.621092 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:32.621093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:32.621094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:32.621097 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:32.621111 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:32.621115 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:32.621171 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:32.621266 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 23:07:32.621286 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:07:32.621639 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.616162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43057 (local address 127.15.212.66:47675)
0928 23:07:32.616346 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:32.616387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:32.616972 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:32.617185 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:32.617188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:32.617196 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:32.617815 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.617825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.618712 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.618713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.619575 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.619577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.620056 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.620057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.620224 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.620925 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:32.620925 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:32.620963 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:32.620989 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:32.621188 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:32.621193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:32.621196 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:32.621269 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 23:07:32.621376 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260928 23:07:32.622192 25756 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43057
I20260928 23:07:32.622295 25756 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:32.622469 25756 heartbeater.cc:511] Master 127.15.212.126:43057 requested a full tablet report, sending...
I20260928 23:07:32.622888 25437 ts_manager.cc:194] Registered new tserver with Master: aa002832c7034d769e246bb7ed6ce45d (127.15.212.66:40385)
I20260928 23:07:32.623425 25437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.66:47675
W20260928 23:07:32.699458 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:32.699648 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:32.699669 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:32.700825 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:32.700879 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:32.700904 25759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:32.702286 25759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:32.702517 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:32.702790 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:32.707661 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.707672 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.707755 25759 server_base.cc:1034] running on GCE node
W20260928 23:07:32.707680 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.708132 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:32.708374 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:32.709585 25759 hybrid_clock.cc:648] HybridClock initialized: now 1790636852709573 us; error 29 us; skew 500 ppm
I20260928 23:07:32.710831 25759 webserver.cc:460] Webserver started at http://127.15.212.67:46855/ using document root <none> and password file <none>
I20260928 23:07:32.711088 25759 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:32.711159 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:32.711311 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:32.712401 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "8a1742e6d0cb459fa3cb0037e4613955"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "c555e61dfc5b6c9564007b887079b65e"
server_key_iv: "51f84023c0e6b3b5606906d1c2dcdcd6"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.712826 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "8a1742e6d0cb459fa3cb0037e4613955"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "c555e61dfc5b6c9564007b887079b65e"
server_key_iv: "51f84023c0e6b3b5606906d1c2dcdcd6"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.714156 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:32.714944 25773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.715128 25759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:32.715226 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "8a1742e6d0cb459fa3cb0037e4613955"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "c555e61dfc5b6c9564007b887079b65e"
server_key_iv: "51f84023c0e6b3b5606906d1c2dcdcd6"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.715313 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:32.741999 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:32.742329 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:32.742482 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:32.742724 25759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:32.743057 25759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:32.743109 25759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.743152 25759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:32.743180 25759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.748677 25759 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:34265
I20260928 23:07:32.748725 25886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:34265 every 8 connection(s)
I20260928 23:07:32.749056 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:32.753916 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25759
I20260928 23:07:32.754009 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:32.754237 16209 external_mini_cluster.cc:1428] Setting key ef7fcc37d67146bf4e2a51a25a539c74
I20260928 23:07:32.755076 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.749791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:54611 (local address 127.15.212.126:43057)
0928 23:07:32.749886 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.749888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.750955 (+ 1067us) server_negotiation.cc:407] Connection header received
0928 23:07:32.751413 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.751414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.751424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.751478 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.752068 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.752637 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.753554 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.753732 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.753798 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.754750 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:32.754751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:32.754752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:32.754756 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:32.754769 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:32.754773 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:32.754841 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:32.754954 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 23:07:32.754981 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 23:07:32.755280 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.749785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:43057 (local address 127.15.212.67:54611)
0928 23:07:32.750834 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:32.750868 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:32.751335 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:32.751530 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:32.751533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:32.751540 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:32.752001 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.752005 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.752710 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.752710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.753467 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:32.753469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:32.753832 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:32.753833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:32.753995 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:32.754590 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:32.754590 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:32.754617 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:32.754638 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:32.754844 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:32.754849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:32.754852 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:32.754957 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 23:07:32.755054 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":82,"threads_started":1}
I20260928 23:07:32.755728 25887 heartbeater.cc:347] Connected to a master server at 127.15.212.126:43057
I20260928 23:07:32.755824 25887 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:32.755993 25887 heartbeater.cc:511] Master 127.15.212.126:43057 requested a full tablet report, sending...
I20260928 23:07:32.756325 25437 ts_manager.cc:194] Registered new tserver with Master: 8a1742e6d0cb459fa3cb0037e4613955 (127.15.212.67:34265)
I20260928 23:07:32.756709 25437 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.67:54611
I20260928 23:07:32.765892 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 23:07:32.767673 25476 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:32.767252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.15.212.126:43057)
0928 23:07:32.767305 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.767307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.767412 (+ 105us) server_negotiation.cc:407] Connection header received
0928 23:07:32.767464 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.767464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.767473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.767528 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.767626 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46390: BlockingRecv error: recv got EOF from 127.0.0.1:46390 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:32.767748 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25497
I20260928 23:07:32.772101 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25628
I20260928 23:07:32.776113 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25759
I20260928 23:07:32.780094 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25406
2026-09-28T23:07:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (585 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T23:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:32Z Disabled control of system clock
I20260928 23:07:32.806025 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46383
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 23:07:32.883961 25898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:32.884143 25898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:32.884164 25898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:32.885269 25898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:32.885313 25898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:32.885326 25898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:32.885339 25898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:32.885350 25898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:32.886576 25898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:32.886744 25898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:32.886953 25898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:32.891537 25906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.891541 25903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:32.891532 25904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:32.891608 25898 server_base.cc:1034] running on GCE node
I20260928 23:07:32.891912 25898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:32.892232 25898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:32.893392 25898 hybrid_clock.cc:648] HybridClock initialized: now 1790636852893384 us; error 34 us; skew 500 ppm
I20260928 23:07:32.894654 25898 webserver.cc:460] Webserver started at http://127.15.212.126:34471/ using document root <none> and password file <none>
I20260928 23:07:32.894899 25898 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:32.894970 25898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:32.895104 25898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:32.896107 25898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "cccab97d2baf4a9992222b024ff71834"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "112a1d963e98dbca9f4a9236c1ff6450"
server_key_iv: "85cd5838528727c836ffc38ca014577f"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.896521 25898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "cccab97d2baf4a9992222b024ff71834"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "112a1d963e98dbca9f4a9236c1ff6450"
server_key_iv: "85cd5838528727c836ffc38ca014577f"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.897753 25898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:07:32.898424 25912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:32.898579 25898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:32.898674 25898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "cccab97d2baf4a9992222b024ff71834"
format_stamp: "Formatted at 2026-09-28 23:07:32 on dist-test-slave-x2t8"
server_key: "112a1d963e98dbca9f4a9236c1ff6450"
server_key_iv: "85cd5838528727c836ffc38ca014577f"
server_key_version: "encryptionkey@0"
I20260928 23:07:32.898761 25898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:32.902026 25898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:32.902273 25898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:32.902412 25898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:32.906037 25898 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46383
I20260928 23:07:32.906064 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46383 every 8 connection(s)
I20260928 23:07:32.906365 25898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:32.906951 25965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:32.909547 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834: Bootstrap starting.
I20260928 23:07:32.909662 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25898
I20260928 23:07:32.909773 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:32.909945 16209 external_mini_cluster.cc:1428] Setting key 3b0037bc14b2f1e0b560b81cebd54e7a
I20260928 23:07:32.910250 25965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:32.910800 25965 log.cc:824] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:32.911533 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834: No bootstrap required, opened a new log
I20260928 23:07:32.912503 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:32.910452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.15.212.126:46383)
0928 23:07:32.910655 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:32.910665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:32.910694 (+ 29us) server_negotiation.cc:407] Connection header received
0928 23:07:32.911180 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:32.911187 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:32.911254 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:32.911400 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:32.912045 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:32.912049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:32.912049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:32.912053 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:32.912096 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:32.912111 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:32.912206 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:32.912241 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 23:07:32.912294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260928 23:07:32.913362 25965 raft_consensus.cc:348] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } }
I20260928 23:07:32.913480 25965 raft_consensus.cc:374] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:32.913502 25965 raft_consensus.cc:729] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cccab97d2baf4a9992222b024ff71834, State: Initialized, Role: FOLLOWER
I20260928 23:07:32.913606 25965 consensus_queue.cc:260] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } }
I20260928 23:07:32.913681 25965 raft_consensus.cc:388] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:32.913702 25965 raft_consensus.cc:482] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:32.913728 25965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:32.914439 25965 raft_consensus.cc:504] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } }
I20260928 23:07:32.914520 25965 leader_election.cc:302] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cccab97d2baf4a9992222b024ff71834; no voters:
I20260928 23:07:32.914634 25965 leader_election.cc:288] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:32.914690 25970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:32.914831 25970 raft_consensus.cc:686] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [term 1 LEADER]: Becoming Leader. State: Replica: cccab97d2baf4a9992222b024ff71834, State: Running, Role: LEADER
I20260928 23:07:32.914880 25965 sys_catalog.cc:564] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:32.914916 25970 consensus_queue.cc:237] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } }
I20260928 23:07:32.915315 25971 sys_catalog.cc:455] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cccab97d2baf4a9992222b024ff71834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } } }
I20260928 23:07:32.915345 25972 sys_catalog.cc:455] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cccab97d2baf4a9992222b024ff71834. Latest consensus state: current_term: 1 leader_uuid: "cccab97d2baf4a9992222b024ff71834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cccab97d2baf4a9992222b024ff71834" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46383 } } }
I20260928 23:07:32.915436 25971 sys_catalog.cc:458] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:32.915444 25972 sys_catalog.cc:458] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:32.915654 25979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:32.916131 25979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:32.917491 25979 catalog_manager.cc:1347] Generated new cluster ID: 1b741ee6e3924bd0bc22abedfd73f11b
I20260928 23:07:32.917547 25979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:32.926994 25979 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:32.927146 25979 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:07:32.931877 25979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cccab97d2baf4a9992222b024ff71834: Generated new TSK 0
I20260928 23:07:32.932053 25979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:32.937449 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--builtin_ntp_servers=127.15.212.84:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 23:07:33.022064 25989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:33.022290 25989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:33.022327 25989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:33.023459 25989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:33.023530 25989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:33.023586 25989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:33.024937 25989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:33.025184 25989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:33.025460 25989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:33.030205 25994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:33.030205 25995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:33.030315 25997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:33.030478 25989 server_base.cc:1034] running on GCE node
I20260928 23:07:33.030694 25989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:33.030952 25989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:33.032109 25989 hybrid_clock.cc:648] HybridClock initialized: now 1790636853032097 us; error 30 us; skew 500 ppm
I20260928 23:07:33.033360 25989 webserver.cc:460] Webserver started at http://127.15.212.65:41341/ using document root <none> and password file <none>
I20260928 23:07:33.033586 25989 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:33.033653 25989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:33.033785 25989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:33.034904 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "e785da20656741c38583ed913a88ed39"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "8ef8c29721544554fa4ba421b77356a7"
server_key_iv: "f12cb91a26596aeed26694af2aa36700"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.035276 25989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "e785da20656741c38583ed913a88ed39"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "8ef8c29721544554fa4ba421b77356a7"
server_key_iv: "f12cb91a26596aeed26694af2aa36700"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.036602 25989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:33.037334 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.037523 25989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.037626 25989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "e785da20656741c38583ed913a88ed39"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "8ef8c29721544554fa4ba421b77356a7"
server_key_iv: "f12cb91a26596aeed26694af2aa36700"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.037712 25989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:33.041038 25989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:33.041293 25989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:33.041429 25989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:33.041646 25989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:33.041965 25989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:33.042018 25989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.042071 25989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:33.042099 25989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.048285 25989 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:35845
I20260928 23:07:33.048326 26116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:35845 every 8 connection(s)
I20260928 23:07:33.048699 25989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:07:33.051832 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.049424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:51701 (local address 127.15.212.126:46383)
0928 23:07:33.049476 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:33.049478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:33.049791 (+ 313us) server_negotiation.cc:407] Connection header received
0928 23:07:33.050332 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:33.050333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:33.050338 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:33.050390 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:33.051606 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:33.051607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:33.051607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:33.051610 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:33.051622 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:33.051626 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:33.051671 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:33.051773 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 23:07:33.051784 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:07:33.052011 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.049408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46383 (local address 127.15.212.65:51701)
0928 23:07:33.049605 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:33.049652 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:33.050237 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:33.050462 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:33.050466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:33.050475 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:33.051420 (+ 945us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:33.051421 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:33.051473 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:33.051505 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:33.051690 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:33.051693 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:33.051696 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:33.051772 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:07:33.051826 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260928 23:07:33.052526 26117 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46383
I20260928 23:07:33.052630 26117 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:33.052772 26117 heartbeater.cc:511] Master 127.15.212.126:46383 requested a full tablet report, sending...
I20260928 23:07:33.053000 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 25989
I20260928 23:07:33.053104 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:33.053315 16209 external_mini_cluster.cc:1428] Setting key a4d2e8bd0b7e6f7ed0618e0b9d597c8d
I20260928 23:07:33.053318 25929 ts_manager.cc:194] Registered new tserver with Master: e785da20656741c38583ed913a88ed39 (127.15.212.65:35845)
I20260928 23:07:33.054205 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--builtin_ntp_servers=127.15.212.84:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 23:07:33.133037 26120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:33.133232 26120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:33.133253 26120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:33.134387 26120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:33.134425 26120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:33.134451 26120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:33.135910 26120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:33.136143 26120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:33.136459 26120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:33.141314 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:33.141289 26128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:33.141286 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:33.141482 26120 server_base.cc:1034] running on GCE node
I20260928 23:07:33.141701 26120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:33.141939 26120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:33.143110 26120 hybrid_clock.cc:648] HybridClock initialized: now 1790636853143096 us; error 41 us; skew 500 ppm
I20260928 23:07:33.144255 26120 webserver.cc:460] Webserver started at http://127.15.212.66:42953/ using document root <none> and password file <none>
I20260928 23:07:33.144550 26120 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:33.144619 26120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:33.144763 26120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:33.145843 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "f3bd19cf5abe410a802ae46bede4d7de"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "7e289a867b19e0a2841b97a86565ea4d"
server_key_iv: "5e4b1afaf5fab710e4beb774c8fd8484"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.146214 26120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "f3bd19cf5abe410a802ae46bede4d7de"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "7e289a867b19e0a2841b97a86565ea4d"
server_key_iv: "5e4b1afaf5fab710e4beb774c8fd8484"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.147533 26120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:07:33.148213 26134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.148383 26120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:07:33.148505 26120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "f3bd19cf5abe410a802ae46bede4d7de"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "7e289a867b19e0a2841b97a86565ea4d"
server_key_iv: "5e4b1afaf5fab710e4beb774c8fd8484"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.148582 26120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:33.151758 26120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:33.151997 26120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:33.152144 26120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:33.152349 26120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:33.152659 26120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:33.152712 26120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.152752 26120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:33.152781 26120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.158424 26120 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:38993
I20260928 23:07:33.158493 26247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:38993 every 8 connection(s)
I20260928 23:07:33.158767 26120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:07:33.161667 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.159420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:57091 (local address 127.15.212.126:46383)
0928 23:07:33.159478 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:33.159480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:33.159700 (+ 220us) server_negotiation.cc:407] Connection header received
0928 23:07:33.160156 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:33.160157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:33.160163 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:33.160224 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:33.161394 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:33.161396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:33.161397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:33.161401 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:33.161416 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:33.161420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:33.161484 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:33.161581 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 23:07:33.161596 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 23:07:33.161851 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.159424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46383 (local address 127.15.212.66:57091)
0928 23:07:33.159583 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:33.159613 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:33.160053 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:33.160292 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:33.160297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:33.160311 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:33.161247 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:33.161248 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:33.161273 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:33.161297 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:33.161487 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:33.161491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:33.161494 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:33.161588 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 23:07:33.161653 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260928 23:07:33.162284 26248 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46383
I20260928 23:07:33.162353 26248 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:33.162508 26248 heartbeater.cc:511] Master 127.15.212.126:46383 requested a full tablet report, sending...
I20260928 23:07:33.162858 25929 ts_manager.cc:194] Registered new tserver with Master: f3bd19cf5abe410a802ae46bede4d7de (127.15.212.66:38993)
I20260928 23:07:33.168818 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26120
I20260928 23:07:33.168898 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:33.169032 16209 external_mini_cluster.cc:1428] Setting key 5402b0ac5133ca88ae31bd824f4fc067
I20260928 23:07:33.169895 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--builtin_ntp_servers=127.15.212.84:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 23:07:33.248378 26251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:33.248611 26251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:33.248649 26251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:33.249825 26251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:33.249889 26251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:33.249931 26251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:33.251335 26251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:33.251585 26251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:33.251868 26251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:33.257205 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:33.257264 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:33.257285 26251 server_base.cc:1034] running on GCE node
W20260928 23:07:33.257205 26259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:33.257663 26251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:33.257853 26251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:33.259037 26251 hybrid_clock.cc:648] HybridClock initialized: now 1790636853259023 us; error 29 us; skew 500 ppm
I20260928 23:07:33.260185 26251 webserver.cc:460] Webserver started at http://127.15.212.67:41117/ using document root <none> and password file <none>
I20260928 23:07:33.260413 26251 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:33.260545 26251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:33.260694 26251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:33.261719 26251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "3fd2e86d69574773891394ac20a650b1"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "0186ba07d60071294f63cd0902d0bb53"
server_key_iv: "74b1785289017ad294ab4e4945f5a317"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.262089 26251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "3fd2e86d69574773891394ac20a650b1"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "0186ba07d60071294f63cd0902d0bb53"
server_key_iv: "74b1785289017ad294ab4e4945f5a317"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.263351 26251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:33.264036 26265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.264204 26251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:33.264299 26251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "3fd2e86d69574773891394ac20a650b1"
format_stamp: "Formatted at 2026-09-28 23:07:33 on dist-test-slave-x2t8"
server_key: "0186ba07d60071294f63cd0902d0bb53"
server_key_iv: "74b1785289017ad294ab4e4945f5a317"
server_key_version: "encryptionkey@0"
I20260928 23:07:33.264384 26251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:33.267591 26251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:33.267832 26251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:33.267982 26251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:33.268224 26251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:33.268595 26251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:33.268647 26251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.268687 26251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:33.268713 26251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:33.274209 26251 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:46423
I20260928 23:07:33.274248 26378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:46423 every 8 connection(s)
I20260928 23:07:33.274554 26251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:33.277247 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.275186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:41057 (local address 127.15.212.126:46383)
0928 23:07:33.275246 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:33.275248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:33.275466 (+ 218us) server_negotiation.cc:407] Connection header received
0928 23:07:33.275890 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:33.275891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:33.275896 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:33.275949 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:33.277062 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:33.277064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:33.277064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:07:33.277067 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:33.277081 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:07:33.277085 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:33.277133 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:33.277187 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 23:07:33.277198 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:07:33.277405 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:33.275178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:46383 (local address 127.15.212.67:41057)
0928 23:07:33.275348 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:33.275384 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:33.275804 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:33.275994 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:33.275997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:33.276004 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:33.276929 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:07:33.276929 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:33.276954 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:07:33.276980 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:33.277128 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:33.277131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:33.277133 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:07:33.277181 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:07:33.277224 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260928 23:07:33.277877 26379 heartbeater.cc:347] Connected to a master server at 127.15.212.126:46383
I20260928 23:07:33.277947 26379 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:33.278103 26379 heartbeater.cc:511] Master 127.15.212.126:46383 requested a full tablet report, sending...
I20260928 23:07:33.278452 25929 ts_manager.cc:194] Registered new tserver with Master: 3fd2e86d69574773891394ac20a650b1 (127.15.212.67:46423)
I20260928 23:07:33.283964 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26251
I20260928 23:07:33.284044 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:33.284214 16209 external_mini_cluster.cc:1428] Setting key 2bac902dfc2a5b036549e72328fa9179
I20260928 23:07:33.285125 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 23:07:33.286819 25968 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:07:33.286432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.15.212.126:46383)
0928 23:07:33.286493 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:33.286494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:33.286574 (+ 80us) server_negotiation.cc:407] Connection header received
0928 23:07:33.286618 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:33.286619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:33.286629 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:33.286699 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:33.286779 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34080: BlockingRecv error: recv got EOF from 127.0.0.1:34080 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:07:33.286897 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25989
I20260928 23:07:33.291092 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26120
I20260928 23:07:33.295258 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26251
I20260928 23:07:33.299044 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 25898
2026-09-28T23:07:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (519 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:33 dist-test-slave-x2t8 krb5kdc[26389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:33 dist-test-slave-x2t8 krb5kdc[26389](info): set up 2 sockets
Sep 28 23:07:33 dist-test-slave-x2t8 krb5kdc[26389](info): commencing operation
krb5kdc: starting...
W20260928 23:07:34.329478 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:07:34.448071 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42021
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:42021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:34.526093 26402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:34.526330 26402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:34.526367 26402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:34.527510 26402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:07:34.527578 26402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:34.527601 26402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:34.527622 26402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:07:34.527652 26402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:07:34.529165 26402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:42021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:34.529436 26402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:34.529664 26402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:34.534359 26408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:34.534380 26407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:34.534372 26410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:34.534727 26402 server_base.cc:1034] running on GCE node
I20260928 23:07:34.534973 26402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:34.535285 26402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:34.536480 26402 hybrid_clock.cc:648] HybridClock initialized: now 1790636854536474 us; error 37 us; skew 500 ppm
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:34.539038 26402 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:07:34.539472 26402 webserver.cc:460] Webserver started at http://127.15.212.126:38187/ using document root <none> and password file <none>
I20260928 23:07:34.539732 26402 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:34.539811 26402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:34.539952 26402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:34.541062 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "8c17ccc0a04847ef9a91d15ab420a737"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "c9e8515ff9235a7a43537785866076be"
server_key_iv: "3ae317615dcb25925badf88ff7f6cec3"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.541417 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "8c17ccc0a04847ef9a91d15ab420a737"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "c9e8515ff9235a7a43537785866076be"
server_key_iv: "3ae317615dcb25925badf88ff7f6cec3"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.542724 26402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:34.543421 26417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.543622 26402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:34.543725 26402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "8c17ccc0a04847ef9a91d15ab420a737"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "c9e8515ff9235a7a43537785866076be"
server_key_iv: "3ae317615dcb25925badf88ff7f6cec3"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.543795 26402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:34.557518 26402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:34.559248 26402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:34.559407 26402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:34.563225 26402 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:42021
I20260928 23:07:34.563272 26469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:42021 every 8 connection(s)
I20260928 23:07:34.563623 26402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:07:34.564648 26470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:34.566807 26470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: Bootstrap starting.
I20260928 23:07:34.567378 26470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:34.567667 26470 log.cc:824] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:34.568370 26470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: No bootstrap required, opened a new log
I20260928 23:07:34.569909 26470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } }
I20260928 23:07:34.570037 26470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:34.570060 26470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c17ccc0a04847ef9a91d15ab420a737, State: Initialized, Role: FOLLOWER
I20260928 23:07:34.570232 26470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } }
I20260928 23:07:34.570297 26470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:07:34.570317 26470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:07:34.570371 26470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:34.571127 26470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } }
I20260928 23:07:34.571209 26470 leader_election.cc:302] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c17ccc0a04847ef9a91d15ab420a737; no voters:
I20260928 23:07:34.571382 26470 leader_election.cc:288] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:07:34.571552 26474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } } }
I20260928 23:07:34.571578 26473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:34.571617 26470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:07:34.571632 26474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:07:34.571720 26473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [term 1 LEADER]: Becoming Leader. State: Replica: 8c17ccc0a04847ef9a91d15ab420a737, State: Running, Role: LEADER
I20260928 23:07:34.571851 26473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } }
I20260928 23:07:34.572211 26474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c17ccc0a04847ef9a91d15ab420a737. Latest consensus state: current_term: 1 leader_uuid: "8c17ccc0a04847ef9a91d15ab420a737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c17ccc0a04847ef9a91d15ab420a737" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42021 } } }
I20260928 23:07:34.572294 26474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737 [sys.catalog]: This master's current role is: LEADER
I20260928 23:07:34.572510 26482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:07:34.572628 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26402
I20260928 23:07:34.572734 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:07:34.572937 16209 external_mini_cluster.cc:1428] Setting key e3c27b75d309705069795dafac4a5c94
I20260928 23:07:34.573105 26482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:07:34.574644 26482 catalog_manager.cc:1347] Generated new cluster ID: 8de3e85ad91048eeb49daf1399fb76a7
I20260928 23:07:34.574700 26482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:07:34.583487 26482 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:07:34.584167 26482 catalog_manager.cc:1504] Loading token signing keys...
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:34.587395 26482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: Generated new TSK 0
I20260928 23:07:34.587605 26482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:07:34.589879 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:34.573368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.15.212.126:42021)
0928 23:07:34.580615 (+ 7247us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:34.580622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:34.580650 (+ 28us) server_negotiation.cc:407] Connection header received
0928 23:07:34.580753 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:34.580758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:34.580798 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:34.580974 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:34.581606 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.582736 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.583686 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.584114 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.584246 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:34.586542 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:34.586546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:34.586546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:34.586557 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:34.588326 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.588775 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.588777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.588778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.588871 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.589084 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.589084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.589085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.589293 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:34.589431 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:34.589561 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 23:07:34.589708 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7183,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:07:34.618844 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:34.698086 26496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:34.698311 26496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:34.698348 26496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:34.699473 26496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:34.699540 26496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:34.699584 26496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:07:34.701603 26496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:34.701853 26496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:34.702080 26496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:34.706739 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:34.706737 26501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:34.706785 26496 server_base.cc:1034] running on GCE node
W20260928 23:07:34.706750 26504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:34.707089 26496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:34.707346 26496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:34.708494 26496 hybrid_clock.cc:648] HybridClock initialized: now 1790636854708484 us; error 40 us; skew 500 ppm
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:34.710865 26496 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:07:34.711310 26496 webserver.cc:460] Webserver started at http://127.15.212.65:39863/ using document root <none> and password file <none>
I20260928 23:07:34.711539 26496 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:34.711603 26496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:34.711735 26496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:34.712780 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "02ce3b05ceb549509ba8ff26613e6837"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "e102e5293f9c42b018c976c6e9bee866"
server_key_iv: "e7b064cfa6d53fab36652336d3e37097"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.713129 26496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "02ce3b05ceb549509ba8ff26613e6837"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "e102e5293f9c42b018c976c6e9bee866"
server_key_iv: "e7b064cfa6d53fab36652336d3e37097"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.714365 26496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:34.715054 26511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.715229 26496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:34.715323 26496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "02ce3b05ceb549509ba8ff26613e6837"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "e102e5293f9c42b018c976c6e9bee866"
server_key_iv: "e7b064cfa6d53fab36652336d3e37097"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.715404 26496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:34.726068 26496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:34.727777 26496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:34.727934 26496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:34.728153 26496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:34.728489 26496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:34.728547 26496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.728588 26496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:34.728617 26496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.734477 26496 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:38795
I20260928 23:07:34.734542 26624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:38795 every 8 connection(s)
I20260928 23:07:34.734862 26496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:34.743649 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26496
I20260928 23:07:34.743748 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:07:34.743765 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:34.735580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:56995 (local address 127.15.212.126:42021)
0928 23:07:34.735627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:34.735630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:34.735882 (+ 252us) server_negotiation.cc:407] Connection header received
0928 23:07:34.736286 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:34.736287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:34.736298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:34.736356 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:34.737288 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.737813 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.738843 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.739043 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.739102 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:34.741139 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:34.741141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:34.741141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:34.741147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:34.742588 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.742917 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.742918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.742918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.742967 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.743178 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.743178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.743179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.743274 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:34.743323 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:34.743559 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 23:07:34.743655 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:07:34.743827 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:34.735531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42021 (local address 127.15.212.65:56995)
0928 23:07:34.735697 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:34.735737 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:34.736198 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:34.736405 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:34.736408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:34.737006 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:34.737195 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:34.737200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.737894 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.737894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:34.738749 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:34.738752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.739119 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.739119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:34.739221 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:34.739679 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:34.739702 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:34.741052 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:34.742658 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:34.742662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:34.742664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:34.742855 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:34.743035 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:34.743036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:34.743036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:34.743124 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:34.743328 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:34.743332 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:34.743432 (+ 100us) client_negotiation.cc:769] Sending connection context
0928 23:07:34.743510 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:07:34.743614 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260928 23:07:34.743961 16209 external_mini_cluster.cc:1428] Setting key cb28cf0315b6689a32e35cecc394c24c
I20260928 23:07:34.744359 26625 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42021
I20260928 23:07:34.744488 26625 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:34.744678 26625 heartbeater.cc:511] Master 127.15.212.126:42021 requested a full tablet report, sending...
I20260928 23:07:34.745193 26434 ts_manager.cc:194] Registered new tserver with Master: 02ce3b05ceb549509ba8ff26613e6837 (127.15.212.65:38795)
I20260928 23:07:34.745839 26434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:56995
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:07:34.767593 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:34.846082 26630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:34.846261 26630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:34.846283 26630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:34.847430 26630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:34.847470 26630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:34.847494 26630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:07:34.848837 26630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:34.849074 26630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:34.849290 26630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:34.854153 26636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:34.854252 26630 server_base.cc:1034] running on GCE node
W20260928 23:07:34.854182 26635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:34.854163 26638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:34.854574 26630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:34.854853 26630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:34.856014 26630 hybrid_clock.cc:648] HybridClock initialized: now 1790636854856001 us; error 24 us; skew 500 ppm
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:34.858570 26630 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:07:34.859025 26630 webserver.cc:460] Webserver started at http://127.15.212.66:43261/ using document root <none> and password file <none>
I20260928 23:07:34.859293 26630 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:34.859364 26630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:34.859508 26630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:34.860579 26630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "e355d7b2f49449198d90624282ddd4f2"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "7c5c1bd9127c7124039af8b95919b130"
server_key_iv: "6f8ae5ede72d6c6b2e5826313eef9a2d"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.860916 26630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "e355d7b2f49449198d90624282ddd4f2"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "7c5c1bd9127c7124039af8b95919b130"
server_key_iv: "6f8ae5ede72d6c6b2e5826313eef9a2d"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.862221 26630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:07:34.862957 26645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.863122 26630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:34.863220 26630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "e355d7b2f49449198d90624282ddd4f2"
format_stamp: "Formatted at 2026-09-28 23:07:34 on dist-test-slave-x2t8"
server_key: "7c5c1bd9127c7124039af8b95919b130"
server_key_iv: "6f8ae5ede72d6c6b2e5826313eef9a2d"
server_key_version: "encryptionkey@0"
I20260928 23:07:34.863297 26630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:34.876582 26630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:34.878779 26630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:34.878948 26630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:34.879184 26630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:34.879474 26630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:34.879526 26630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.879567 26630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:34.879616 26630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:34.885250 26630 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:41405
I20260928 23:07:34.885318 26758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:41405 every 8 connection(s)
I20260928 23:07:34.885627 26630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:07:34 dist-test-slave-x2t8 krb5kdc[26389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:34.891780 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26630
I20260928 23:07:34.891903 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:07:34.892138 16209 external_mini_cluster.cc:1428] Setting key 567631f338565b0e29b0d29373339b1a
I20260928 23:07:34.894486 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:34.886358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:55965 (local address 127.15.212.126:42021)
0928 23:07:34.886425 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:34.886427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:34.886600 (+ 173us) server_negotiation.cc:407] Connection header received
0928 23:07:34.887047 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:34.887048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:34.887059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:34.887115 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:34.887932 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.888402 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.889455 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.889652 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.889703 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:34.891521 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:34.891523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:34.891523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:34.891529 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:34.893259 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.893607 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.893608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.893608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.893666 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:34.893886 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:34.893886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:34.893887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:34.894026 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:34.894093 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:34.894325 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 23:07:34.894400 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:07:34.894659 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:34.886320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42021 (local address 127.15.212.66:55965)
0928 23:07:34.886482 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:34.886514 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:34.886966 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:34.887171 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:34.887173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:34.887686 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:34.887859 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:34.887864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.888585 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.888586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:34.889385 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:34.889387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:34.889721 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:34.889722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:34.889809 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:34.890185 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:34.890192 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:34.891446 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:34.893330 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:34.893335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:34.893336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:34.893533 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:34.893735 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:34.893736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:34.893736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:34.893823 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:34.894087 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:34.894090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:34.894193 (+ 103us) client_negotiation.cc:769] Sending connection context
0928 23:07:34.894299 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 23:07:34.894438 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260928 23:07:34.895215 26759 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42021
I20260928 23:07:34.895327 26759 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:34.895498 26759 heartbeater.cc:511] Master 127.15.212.126:42021 requested a full tablet report, sending...
I20260928 23:07:34.895917 26434 ts_manager.cc:194] Registered new tserver with Master: e355d7b2f49449198d90624282ddd4f2 (127.15.212.66:41405)
I20260928 23:07:34.896436 26434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:55965
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:07:34.917320 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:07:34.996162 26764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:07:34.996383 26764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:07:34.996419 26764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:07:34.997555 26764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:07:34.997622 26764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:07:34.997664 26764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:07:34.999094 26764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:07:34.999348 26764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:07:34.999567 26764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:07:35.004328 26770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:35.004393 26764 server_base.cc:1034] running on GCE node
W20260928 23:07:35.004334 26772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:07:35.004334 26769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:07:35.004779 26764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:07:35.005059 26764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:07:35.006222 26764 hybrid_clock.cc:648] HybridClock initialized: now 1790636855006222 us; error 38 us; skew 500 ppm
Sep 28 23:07:35 dist-test-slave-x2t8 krb5kdc[26389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:07:35.008610 26764 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:07:35.009038 26764 webserver.cc:460] Webserver started at http://127.15.212.67:36003/ using document root <none> and password file <none>
I20260928 23:07:35.009261 26764 fs_manager.cc:359] Metadata directory not provided
I20260928 23:07:35.009325 26764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:07:35.009452 26764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:07:35.010404 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "82f17a9dd7c84d168a4e457c6d9ba071"
format_stamp: "Formatted at 2026-09-28 23:07:35 on dist-test-slave-x2t8"
server_key: "d486549b8712583afca146ffda369260"
server_key_iv: "1ac26fae424cf79ed283e7882cdacb17"
server_key_version: "encryptionkey@0"
I20260928 23:07:35.010746 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "82f17a9dd7c84d168a4e457c6d9ba071"
format_stamp: "Formatted at 2026-09-28 23:07:35 on dist-test-slave-x2t8"
server_key: "d486549b8712583afca146ffda369260"
server_key_iv: "1ac26fae424cf79ed283e7882cdacb17"
server_key_version: "encryptionkey@0"
I20260928 23:07:35.011996 26764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:07:35.012705 26779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:35.012872 26764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:07:35.012965 26764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "82f17a9dd7c84d168a4e457c6d9ba071"
format_stamp: "Formatted at 2026-09-28 23:07:35 on dist-test-slave-x2t8"
server_key: "d486549b8712583afca146ffda369260"
server_key_iv: "1ac26fae424cf79ed283e7882cdacb17"
server_key_version: "encryptionkey@0"
I20260928 23:07:35.013057 26764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:07:35.028405 26764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:07:35.030164 26764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:07:35.030328 26764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:07:35.030543 26764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:07:35.030833 26764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:07:35.030889 26764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:35.030927 26764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:07:35.030956 26764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:07:35.036580 26764 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:46549
I20260928 23:07:35.036643 26892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:46549 every 8 connection(s)
I20260928 23:07:35.036960 26764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:07:35.041723 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 26764
I20260928 23:07:35.041818 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:07:35.042009 16209 external_mini_cluster.cc:1428] Setting key feac7eb1ad387210d68b6cd5f01cb84a
Sep 28 23:07:35 dist-test-slave-x2t8 krb5kdc[26389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:07:35.047966 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.037661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57365 (local address 127.15.212.126:42021)
0928 23:07:35.037705 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.037708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.037873 (+ 165us) server_negotiation.cc:407] Connection header received
0928 23:07:35.038373 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.038374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.038386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.038436 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:35.039389 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.040095 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.041535 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.041864 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.041936 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.044824 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:35.044825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:35.044826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:35.044833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:35.046424 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:35.046896 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:35.046897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:35.046898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:35.046979 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:35.047233 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:35.047234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:35.047235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:35.047368 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:35.047448 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.047795 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.047879 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:07:35.048179 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.037623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42021 (local address 127.15.212.67:57365)
0928 23:07:35.037753 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.037796 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.038265 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.038498 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.038501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.039033 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:07:35.039271 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.039277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.040200 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.040201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.041428 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.041430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.042001 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.042002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.042157 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.042729 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:07:35.042739 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:07:35.044726 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:07:35.046509 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:35.046514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:35.046516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:35.046817 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:35.047046 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:07:35.047047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:07:35.047047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:07:35.047165 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:07:35.047476 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:07:35.047482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:07:35.047624 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.047757 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.047922 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260928 23:07:35.048573 26893 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42021
I20260928 23:07:35.048678 26893 heartbeater.cc:464] Registering TS with master...
I20260928 23:07:35.048840 26893 heartbeater.cc:511] Master 127.15.212.126:42021 requested a full tablet report, sending...
I20260928 23:07:35.049172 26434 ts_manager.cc:194] Registered new tserver with Master: 82f17a9dd7c84d168a4e457c6d9ba071 (127.15.212.67:46549)
I20260928 23:07:35.049499 26434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:57365
I20260928 23:07:35.055553 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:07:35.062117 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.056771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.15.212.126:42021)
0928 23:07:35.056840 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.056842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.056893 (+ 51us) server_negotiation.cc:407] Connection header received
0928 23:07:35.056942 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.056943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.056951 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.057011 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:07:35.057522 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.057988 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.058668 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.058845 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.058893 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.059942 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:07:35.059943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:07:35.059943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:07:35.059948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:07:35.061167 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:35.061463 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:35.061463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:35.061463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:35.061511 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:07:35.061670 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:07:35.061671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:07:35.061671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:07:35.061789 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:07:35.061860 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.061950 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.062014 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 23:07:35.063659 26434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:07:35.065312 26434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:07:35.071501 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:46549 (local address 127.0.0.1:55194)
0928 23:07:35.067618 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.067661 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.067732 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.067904 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.067906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.067909 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:35.067966 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.067967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069221 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.069223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.070377 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.070380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071158 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.071288 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071302 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.071374 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.071422 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":61,"threads_started":1}
I20260928 23:07:35.071527 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.15.212.67:46549)
0928 23:07:35.067760 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.067764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.067774 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:07:35.067809 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.067811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.067824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.067895 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:35.068033 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069115 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.070456 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071074 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071158 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071270 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.071394 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.071433 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260928 23:07:35.071755 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41405 (local address 127.0.0.1:54806)
0928 23:07:35.067638 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.067668 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.067729 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.068256 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.068256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.068259 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:35.068325 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.068326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069510 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.069510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.070593 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.070595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071506 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.071573 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071578 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.071614 (+ 36us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.071671 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":52,"threads_started":1}
I20260928 23:07:35.071868 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38795 (local address 127.0.0.1:55330)
0928 23:07:35.067833 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.067843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.067879 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.068698 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.068699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.068701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:35.068735 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.068737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069767 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.069768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.070776 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.070777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071626 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.071682 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071686 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.071730 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.071767 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":65,"threads_started":1}
I20260928 23:07:35.072086 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.15.212.65:38795)
0928 23:07:35.068543 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.068548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.068557 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:35.068592 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.068593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.068605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.068686 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:35.068797 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069667 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.070849 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071462 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071592 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.071965 (+ 373us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.072013 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":85,"threads_started":1}
I20260928 23:07:35.072122 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.067472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.15.212.66:41405)
0928 23:07:35.067980 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.067986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.068001 (+ 15us) server_negotiation.cc:407] Connection header received
0928 23:07:35.068071 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.068074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.068090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.068176 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:35.068387 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.069402 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.070675 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.071415 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.071921 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.071981 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.072008 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.072046 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":77,"threads_started":1}
I20260928 23:07:35.072374 26827 tablet_service.cc:1467] Processing CreateTablet for tablet f40a36f6501942d28ace855dd6351e6a (DEFAULT_TABLE table=test-table [id=fc4f2f9b60604afbaaf5bf34b65334a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:35.072759 26827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40a36f6501942d28ace855dd6351e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:35.072747 26559 tablet_service.cc:1467] Processing CreateTablet for tablet f40a36f6501942d28ace855dd6351e6a (DEFAULT_TABLE table=test-table [id=fc4f2f9b60604afbaaf5bf34b65334a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:35.072747 26693 tablet_service.cc:1467] Processing CreateTablet for tablet f40a36f6501942d28ace855dd6351e6a (DEFAULT_TABLE table=test-table [id=fc4f2f9b60604afbaaf5bf34b65334a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:07:35.073036 26693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40a36f6501942d28ace855dd6351e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:35.073036 26559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f40a36f6501942d28ace855dd6351e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:07:35.075636 26907 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Bootstrap starting.
I20260928 23:07:35.076140 26908 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: Bootstrap starting.
I20260928 23:07:35.076265 26907 tablet_bootstrap.cc:654] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:35.076876 26908 tablet_bootstrap.cc:654] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:35.076886 26907 log.cc:824] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:35.077417 26908 log.cc:824] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:35.077673 26907 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: No bootstrap required, opened a new log
I20260928 23:07:35.077744 26907 ts_tablet_manager.cc:1397] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:07:35.077755 26909 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Bootstrap starting.
I20260928 23:07:35.078128 26908 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: No bootstrap required, opened a new log
I20260928 23:07:35.078212 26908 ts_tablet_manager.cc:1397] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:07:35.078351 26909 tablet_bootstrap.cc:654] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Neither blocks nor log segments found. Creating new log.
I20260928 23:07:35.078876 26909 log.cc:824] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Log is configured to *not* fsync() on all Append() calls
I20260928 23:07:35.079594 26909 tablet_bootstrap.cc:492] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: No bootstrap required, opened a new log
I20260928 23:07:35.079691 26909 ts_tablet_manager.cc:1397] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:07:35.079865 26907 raft_consensus.cc:348] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.080027 26907 raft_consensus.cc:374] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:35.080062 26907 raft_consensus.cc:729] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f17a9dd7c84d168a4e457c6d9ba071, State: Initialized, Role: FOLLOWER
I20260928 23:07:35.080065 26908 raft_consensus.cc:348] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.080180 26907 consensus_queue.cc:260] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.080224 26908 raft_consensus.cc:374] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:35.080269 26908 raft_consensus.cc:729] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ce3b05ceb549509ba8ff26613e6837, State: Initialized, Role: FOLLOWER
I20260928 23:07:35.080426 26907 ts_tablet_manager.cc:1428] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:07:35.080406 26908 consensus_queue.cc:260] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.080561 26893 heartbeater.cc:503] Master 127.15.212.126:42021 was elected leader, sending a full tablet report...
I20260928 23:07:35.080709 26908 ts_tablet_manager.cc:1428] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:07:35.080734 26625 heartbeater.cc:503] Master 127.15.212.126:42021 was elected leader, sending a full tablet report...
I20260928 23:07:35.081769 26909 raft_consensus.cc:348] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.081909 26909 raft_consensus.cc:374] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:07:35.081931 26909 raft_consensus.cc:729] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e355d7b2f49449198d90624282ddd4f2, State: Initialized, Role: FOLLOWER
I20260928 23:07:35.082000 26909 consensus_queue.cc:260] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.082186 26909 ts_tablet_manager.cc:1428] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:07:35.082257 26759 heartbeater.cc:503] Master 127.15.212.126:42021 was elected leader, sending a full tablet report...
I20260928 23:07:35.127758 26913 raft_consensus.cc:482] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:07:35.127883 26913 raft_consensus.cc:504] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.128213 26913 leader_election.cc:288] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e355d7b2f49449198d90624282ddd4f2 (127.15.212.66:41405), 02ce3b05ceb549509ba8ff26613e6837 (127.15.212.65:38795)
I20260928 23:07:35.131090 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.128489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:41783 (local address 127.15.212.66:41405)
0928 23:07:35.128534 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.128536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.128547 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:07:35.128577 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.128578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.128590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.128662 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:35.128835 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.129328 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130342 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.130717 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130772 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.130779 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.130963 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.131004 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:07:35.131122 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.128472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:45375 (local address 127.15.212.65:38795)
0928 23:07:35.128516 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.128518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.128649 (+ 131us) server_negotiation.cc:407] Connection header received
0928 23:07:35.128679 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.128679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.128688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.128746 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:07:35.128932 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.129429 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130374 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.130743 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130836 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.130844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.131014 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.131066 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:07:35.131111 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.128371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:41405 (local address 127.15.212.67:41783)
0928 23:07:35.128426 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.128456 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.128499 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.128669 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.128670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.128672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:35.128754 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.128756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.129412 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.129412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.130248 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.130250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.130800 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.130879 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.130882 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.130933 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.130982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":27}
I20260928 23:07:35.131146 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.128383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38795 (local address 127.15.212.67:45375)
0928 23:07:35.128562 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.128572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:07:35.128631 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:07:35.128766 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:07:35.128766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:07:35.128767 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:07:35.128851 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.128853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.129509 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.129509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.130260 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:07:35.130261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.130853 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.130853 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:07:35.130906 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.130909 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:07:35.130987 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:07:35.131019 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
I20260928 23:07:35.131466 26713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40a36f6501942d28ace855dd6351e6a" candidate_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e355d7b2f49449198d90624282ddd4f2" is_pre_election: true
I20260928 23:07:35.131515 26579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40a36f6501942d28ace855dd6351e6a" candidate_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ce3b05ceb549509ba8ff26613e6837" is_pre_election: true
I20260928 23:07:35.131649 26713 raft_consensus.cc:2454] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f17a9dd7c84d168a4e457c6d9ba071 in term 0.
I20260928 23:07:35.131680 26579 raft_consensus.cc:2454] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82f17a9dd7c84d168a4e457c6d9ba071 in term 0.
I20260928 23:07:35.131871 26780 leader_election.cc:302] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02ce3b05ceb549509ba8ff26613e6837, 82f17a9dd7c84d168a4e457c6d9ba071; no voters:
I20260928 23:07:35.131989 26913 raft_consensus.cc:2781] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:07:35.132037 26913 raft_consensus.cc:482] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:07:35.132058 26913 raft_consensus.cc:3037] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:35.132761 26913 raft_consensus.cc:504] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.132896 26913 leader_election.cc:288] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [CANDIDATE]: Term 1 election: Requested vote from peers e355d7b2f49449198d90624282ddd4f2 (127.15.212.66:41405), 02ce3b05ceb549509ba8ff26613e6837 (127.15.212.65:38795)
I20260928 23:07:35.133049 26579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40a36f6501942d28ace855dd6351e6a" candidate_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02ce3b05ceb549509ba8ff26613e6837"
I20260928 23:07:35.133044 26713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f40a36f6501942d28ace855dd6351e6a" candidate_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e355d7b2f49449198d90624282ddd4f2"
I20260928 23:07:35.133116 26579 raft_consensus.cc:3037] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:35.133136 26713 raft_consensus.cc:3037] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:07:35.133849 26713 raft_consensus.cc:2454] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f17a9dd7c84d168a4e457c6d9ba071 in term 1.
I20260928 23:07:35.133924 26579 raft_consensus.cc:2454] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82f17a9dd7c84d168a4e457c6d9ba071 in term 1.
I20260928 23:07:35.134027 26783 leader_election.cc:302] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82f17a9dd7c84d168a4e457c6d9ba071, e355d7b2f49449198d90624282ddd4f2; no voters:
I20260928 23:07:35.134104 26913 raft_consensus.cc:2781] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:07:35.134260 26913 raft_consensus.cc:686] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 1 LEADER]: Becoming Leader. State: Replica: 82f17a9dd7c84d168a4e457c6d9ba071, State: Running, Role: LEADER
I20260928 23:07:35.134356 26913 consensus_queue.cc:237] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } }
I20260928 23:07:35.135185 26433 catalog_manager.cc:5495] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82f17a9dd7c84d168a4e457c6d9ba071 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82f17a9dd7c84d168a4e457c6d9ba071" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 } health_report { overall_health: UNKNOWN } } }
W20260928 23:07:35.136082 26760 tablet.cc:2369] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:07:35.143540 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.141219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.15.212.67:46549)
0928 23:07:35.141275 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:07:35.141277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:07:35.141286 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:07:35.141341 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:07:35.141342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:07:35.141359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:07:35.141428 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:07:35.141571 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.142023 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.142767 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:07:35.142969 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:07:35.143042 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:07:35.143219 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:07:35.143317 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:07:35.143412 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:07:35.143431 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 23:07:35.143448 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 23:07:35.145171 26579 raft_consensus.cc:1261] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 1 FOLLOWER]: Refusing update from remote peer 82f17a9dd7c84d168a4e457c6d9ba071: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:35.145453 26713 raft_consensus.cc:1261] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 1 FOLLOWER]: Refusing update from remote peer 82f17a9dd7c84d168a4e457c6d9ba071: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:07:35.145481 26917 consensus_queue.cc:1035] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02ce3b05ceb549509ba8ff26613e6837" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:35.145777 26917 consensus_queue.cc:1035] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e355d7b2f49449198d90624282ddd4f2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 41405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:07:35.147840 26920 mvcc.cc:205] Tried to move back new op lower bound from 7334448558672310272 to 7334448558631022592. Current Snapshot: MvccSnapshot[applied={T|T < 7334448558672310272}]
I20260928 23:07:35.147966 26921 mvcc.cc:205] Tried to move back new op lower bound from 7334448558672310272 to 7334448558631022592. Current Snapshot: MvccSnapshot[applied={T|T < 7334448558672310272}]
I20260928 23:07:35.150249 26432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:35.150339 26432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:07:35.151083 26432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c17ccc0a04847ef9a91d15ab420a737: Sending DeleteTablet for 3 replicas of tablet f40a36f6501942d28ace855dd6351e6a
I20260928 23:07:35.151383 26827 tablet_service.cc:1514] Processing DeleteTablet for tablet f40a36f6501942d28ace855dd6351e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:55194
I20260928 23:07:35.151409 26559 tablet_service.cc:1514] Processing DeleteTablet for tablet f40a36f6501942d28ace855dd6351e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:55330
I20260928 23:07:35.151408 26693 tablet_service.cc:1514] Processing DeleteTablet for tablet f40a36f6501942d28ace855dd6351e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:07:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:54806
I20260928 23:07:35.151629 26931 tablet_replica.cc:330] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: stopping tablet replica
I20260928 23:07:35.151724 26930 tablet_replica.cc:330] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837: stopping tablet replica
I20260928 23:07:35.151729 26931 raft_consensus.cc:2229] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:07:35.151813 26931 raft_consensus.cc:2258] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:35.151840 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26496
I20260928 23:07:35.151925 26930 raft_consensus.cc:2229] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:35.152051 26930 raft_consensus.cc:2258] T f40a36f6501942d28ace855dd6351e6a P 02ce3b05ceb549509ba8ff26613e6837 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:35.153844 26932 tablet_replica.cc:330] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: stopping tablet replica
I20260928 23:07:35.153941 26932 raft_consensus.cc:2229] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:07:35.154034 26932 raft_consensus.cc:2258] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:07:35.154331 26931 ts_tablet_manager.cc:1905] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:35.155932 26931 ts_tablet_manager.cc:1918] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:35.155944 26932 ts_tablet_manager.cc:1905] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:07:35.156075 26931 log.cc:1196] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/f40a36f6501942d28ace855dd6351e6a
I20260928 23:07:35.156488 26931 ts_tablet_manager.cc:1939] T f40a36f6501942d28ace855dd6351e6a P 82f17a9dd7c84d168a4e457c6d9ba071: Deleting consensus metadata
I20260928 23:07:35.156937 26419 catalog_manager.cc:4841] TS 82f17a9dd7c84d168a4e457c6d9ba071 (127.15.212.67:46549): tablet f40a36f6501942d28ace855dd6351e6a (table test-table [id=fc4f2f9b60604afbaaf5bf34b65334a0]) successfully deleted
I20260928 23:07:35.157575 26932 ts_tablet_manager.cc:1918] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:07:35.157672 26932 log.cc:1196] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/f40a36f6501942d28ace855dd6351e6a
I20260928 23:07:35.157917 26932 ts_tablet_manager.cc:1939] T f40a36f6501942d28ace855dd6351e6a P e355d7b2f49449198d90624282ddd4f2: Deleting consensus metadata
W20260928 23:07:35.158254 26418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:38795 (error 108)
I20260928 23:07:35.158298 26421 catalog_manager.cc:4841] TS e355d7b2f49449198d90624282ddd4f2 (127.15.212.66:41405): tablet f40a36f6501942d28ace855dd6351e6a (table test-table [id=fc4f2f9b60604afbaaf5bf34b65334a0]) successfully deleted
I20260928 23:07:35.158478 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26630
I20260928 23:07:35.159086 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:07:35.158553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:38795 (local address 127.0.0.1:55340)
0928 23:07:35.159009 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0928 23:07:35.159045 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:38795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437}
W20260928 23:07:35.159332 26418 catalog_manager.cc:4584] TS 02ce3b05ceb549509ba8ff26613e6837 (127.15.212.65:38795): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f40a36f6501942d28ace855dd6351e6a: Network error: Client connection negotiation failed: client connection to 127.15.212.65:38795: connect: Connection refused (error 111)
I20260928 23:07:35.163820 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26764
I20260928 23:07:35.168321 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 26402
2026-09-28T23:07:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1869 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:07:35 dist-test-slave-x2t8 krb5kdc[26936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:07:35 dist-test-slave-x2t8 krb5kdc[26936](info): set up 2 sockets
Sep 28 23:07:35 dist-test-slave-x2t8 krb5kdc[26936](info): commencing operation
krb5kdc: starting...
W20260928 23:07:36.201457 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:07:36 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:07:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:07:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangeradmin_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangeradmin_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangerlookup_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangerlookup_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
I20260928 23:07:36.362668 16209 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdR8rxt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/postgres -l logfile start
2026-09-28 23:07:38.811 UTC [26966] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 23:07:38.811 UTC [26966] LOG: listening on IPv4 address "127.15.212.84", port 60499
2026-09-28 23:07:38.815 UTC [26966] LOG: listening on Unix socket "/tmp/.s.PGSQL.60499"
2026-09-28 23:07:38.836 UTC [26969] LOG: database system was shut down at 2026-09-28 23:07:37 UTC
2026-09-28 23:07:38.840 UTC [26966] LOG: database system is ready to accept connections
I20260928 23:07:39.785404 16209 mini_postgres.cc:96] Postgres bound to 60499
2026-09-28 23:07:39.791 UTC [26977] FATAL: database "slave" does not exist
127.15.212.84:60499 - accepting connections
I20260928 23:07:39.791877 16209 mini_ranger.cc:161] Starting Ranger...
I20260928 23:07:39.805704 16209 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 23:07:40.281975 16209 mini_ranger.cc:87] Created ranger Postgres database
I20260928 23:07:40.282038 16209 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin
2026-09-28 23:07:40,551 [I] DB FLAVOR :POSTGRES
2026-09-28 23:07:40,552 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 23:07:40,552 [I] Checking connection..
2026-09-28 23:07:40,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 23:07:40,783 [I] Checking connection passed.
2026-09-28 23:07:40,783 [I] --------- Verifying version history table ---------
2026-09-28 23:07:40,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:07:41,022 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 23:07:41,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:07:41,255 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 23:07:41,256 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 23:07:41,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 23:07:41.479 UTC [27041] WARNING: there is no transaction in progress
2026-09-28 23:07:41,485 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 23:07:41,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:07:41,738 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 23:07:41,738 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 23:07:41,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 23:07:41,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 23:07:42,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x2t8.c.gcp-upstream.internal','N') ;"
2026-09-28 23:07:42,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 23:07:42,710 [I] Table x_portal_user does not exist in database ranger
2026-09-28 23:07:42,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 23:07:42,953 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 23:07:42,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 23:07:43,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 23:07:43,438 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 23:07:43,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 23:07:44.362 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.377 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.393 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.408 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.424 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.442 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.495 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.501 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.514 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.524 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.534 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.547 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.556 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.562 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.568 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:44.574 UTC [27149] WARNING: there is no transaction in progress
2026-09-28 23:07:45,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 23:07:45,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 23:07:45,424 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 23:07:45,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 23:07:45,655 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 23:07:45,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 23:07:45,888 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 23:07:45,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 23:07:46,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 23:07:46,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x2t8.c.gcp-upstream.internal';"
2026-09-28 23:07:46,572 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 23:07:46,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 23:07:46,814 [I] DB_PATCHES have already been applied
I20260928 23:07:46.819679 16209 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdR8rxt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 23:07:46.819741 16209 mini_ranger.cc:193] Using host: 127.15.212.84
I20260928 23:07:46.822724 16209 mini_ranger.cc:227] Ranger admin URL: http://127.15.212.84:50303
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 11:07:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 23:07:47.918738 16209 mini_ranger.cc:160] Time spent starting Ranger: real 8.127s user 0.000s sys 0.008s
Sep 28, 2026 11:08:07 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 11:08:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 11:08:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 11:08:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 11:08:08 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 23:08:09.113358 16209 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangerkms_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/rangerkms_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.212.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.212.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 23:08:09 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636889, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 23:08:09.196774 16209 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 23:08:09.206005 16209 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 23:08:10.047894 16209 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 23:08:10.047977 16209 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms
I20260928 23:08:10.048005 16209 mini_ranger_kms.cc:227] Using postgres at 127.15.212.84:60499
2026-09-28 23:08:10,232 [I] DB FLAVOR :POSTGRES
2026-09-28 23:08:10,232 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 23:08:10,232 [I] Checking connection
2026-09-28 23:08:10,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 23:08:10,500 [I] connection success
2026-09-28 23:08:10,500 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 23:08:10,500 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 23:08:10,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 23:08:10,769 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 23:08:10,770 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 23:08:10,770 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 23:08:10,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:60499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 23:08:11,035 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 23:08:11.497166 16209 mini_ranger_kms.cc:310] Created kms service
I20260928 23:08:11.664752 16209 mini_ranger_kms.cc:326] Created kudu user
I20260928 23:08:11.706382 16209 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 23:08:11 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636889, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:12.098482 16209 mini_ranger_kms.cc:384] Added ranger policy
I20260928 23:08:12.098600 16209 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 23:08:12.098623 16209 mini_ranger_kms.cc:240] Using host: 127.15.212.84
I20260928 23:08:12.102602 16209 mini_ranger_kms.cc:276] Ranger KMS PID: 27329
I20260928 23:08:12.102725 16209 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.212.84:39091
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 11:08:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 23:08:14.308468 16209 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.112s user 0.009s sys 0.158s
I20260928 23:08:14.308563 16209 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 23:08:14.308648 16209 mini_ranger_kms.cc:401] 127.15.212.84:39091/kms/v1/keys
Sep 28 23:08:14 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790636894, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 23:08:14 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636894, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
Sep 28 23:08:15 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636894, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
Sep 28 23:08:17 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:08:17.546830 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39031
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:39031
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:39091/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:08:17.633888 27380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:08:17.634089 27380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:08:17.634110 27380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:08:17.635224 27380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:08:17.635263 27380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:08:17.635277 27380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:08:17.635288 27380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:08:17.635300 27380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:08:17.636849 27380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.212.84:39091/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:39031
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:08:17.637059 27380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:08:17.637333 27380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:08:17.643414 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:08:17.643544 27380 server_base.cc:1034] running on GCE node
W20260928 23:08:17.643414 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:17.643646 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:08:17.643860 27380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:08:17.644181 27380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:08:17.645344 27380 hybrid_clock.cc:648] HybridClock initialized: now 1790636897645329 us; error 31 us; skew 500 ppm
Sep 28 23:08:17 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636897, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:17.654065 27380 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:08:17.654541 27380 webserver.cc:460] Webserver started at http://127.15.212.126:36461/ using document root <none> and password file <none>
I20260928 23:08:17.654824 27380 fs_manager.cc:359] Metadata directory not provided
I20260928 23:08:17.654891 27380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:08:17.655042 27380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:08:17 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636897, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:17.698421 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "6c927a5ed02e452eb0a4a46cef2535d8"
format_stamp: "Formatted at 2026-09-28 23:08:17 on dist-test-slave-x2t8"
server_key: "4c3edd537bd22ac18d7df9ee9908d093"
server_key_iv: "b6b86eaa8f7014d65fa232cffd13782e"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:17.698915 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "6c927a5ed02e452eb0a4a46cef2535d8"
format_stamp: "Formatted at 2026-09-28 23:08:17 on dist-test-slave-x2t8"
server_key: "4c3edd537bd22ac18d7df9ee9908d093"
server_key_iv: "b6b86eaa8f7014d65fa232cffd13782e"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:17.700861 27380 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260928 23:08:17.714457 27395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:17.714694 27380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:08:17.714792 27380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "6c927a5ed02e452eb0a4a46cef2535d8"
format_stamp: "Formatted at 2026-09-28 23:08:17 on dist-test-slave-x2t8"
server_key: "4c3edd537bd22ac18d7df9ee9908d093"
server_key_iv: "b6b86eaa8f7014d65fa232cffd13782e"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:17.714881 27380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:08:17.727788 27380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:08:17.729820 27380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:08:17.730005 27380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:08:17.735260 27380 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:39031
I20260928 23:08:17.735282 27447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:39031 every 8 connection(s)
I20260928 23:08:17.735754 27380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:08:17.736689 27448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:08:17.739025 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: Bootstrap starting.
I20260928 23:08:17.739589 27448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: Neither blocks nor log segments found. Creating new log.
I20260928 23:08:17.739890 27448 log.cc:824] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: Log is configured to *not* fsync() on all Append() calls
I20260928 23:08:17.740653 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: No bootstrap required, opened a new log
I20260928 23:08:17.741896 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 27380
I20260928 23:08:17.741994 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:08:17.742203 16209 external_mini_cluster.cc:1428] Setting key 6614f77951f800eba757d3c4b322fab9
I20260928 23:08:17.743094 27448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } }
I20260928 23:08:17.743281 27448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:08:17.743316 27448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c927a5ed02e452eb0a4a46cef2535d8, State: Initialized, Role: FOLLOWER
I20260928 23:08:17.743494 27448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } }
I20260928 23:08:17.743577 27448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:08:17.743606 27448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:08:17.743681 27448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:08:17.744397 27448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } }
I20260928 23:08:17.744693 27448 leader_election.cc:302] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c927a5ed02e452eb0a4a46cef2535d8; no voters:
I20260928 23:08:17.744920 27448 leader_election.cc:288] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:08:17.744961 27453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:08:17.745167 27453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [term 1 LEADER]: Becoming Leader. State: Replica: 6c927a5ed02e452eb0a4a46cef2535d8, State: Running, Role: LEADER
I20260928 23:08:17.745280 27448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:08:17.745291 27453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } }
I20260928 23:08:17.745707 27454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c927a5ed02e452eb0a4a46cef2535d8. Latest consensus state: current_term: 1 leader_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } } }
I20260928 23:08:17.745922 27454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [sys.catalog]: This master's current role is: LEADER
I20260928 23:08:17.745867 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c927a5ed02e452eb0a4a46cef2535d8" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39031 } } }
I20260928 23:08:17.747241 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8 [sys.catalog]: This master's current role is: LEADER
I20260928 23:08:17.747126 27448 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 28 23:08:17 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:08:17.752645 27468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:08:17.755245 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:17.742742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.15.212.126:39031)
0928 23:08:17.743173 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:17.743182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:17.743210 (+ 28us) server_negotiation.cc:407] Connection header received
0928 23:08:17.743317 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:17.743342 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:17.743399 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:17.743531 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:08:17.744423 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:17.745992 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:17.747153 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:17.747879 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:17.747988 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:17.751133 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:08:17.751153 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:08:17.751153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:08:17.751200 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:08:17.753532 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:17.753951 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:17.753954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:17.753954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:17.754050 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:17.754295 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:17.754295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:17.754296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:17.754606 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:08:17.754762 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:17.754879 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 23:08:17.755021 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":89,"threads_started":1}
I20260928 23:08:17.755903 27468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:08:17.758473 27468 catalog_manager.cc:1347] Generated new cluster ID: ad4b9b9cc07c4e4789583b7090947aaf
I20260928 23:08:17.758576 27468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:08:17.772583 27468 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:08:17.773324 27468 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:08:17.779731 27468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: Generated new TSK 0
I20260928 23:08:17.779955 27468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:08:18.509661 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:39091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:08:18.620416 27493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:08:18.620682 27493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:08:18.620720 27493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:08:18.621879 27493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:08:18.621946 27493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:08:18.621989 27493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:08:18.623426 27493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.212.84:39091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:08:18.623675 27493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:08:18.623927 27493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:08:18.629000 27498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:18.629281 27501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:18.628985 27499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:08:18.629464 27493 server_base.cc:1034] running on GCE node
I20260928 23:08:18.629662 27493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:08:18.629918 27493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:08:18.631083 27493 hybrid_clock.cc:648] HybridClock initialized: now 1790636898631070 us; error 25 us; skew 500 ppm
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:18.633781 27493 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:08:18.634310 27493 webserver.cc:460] Webserver started at http://127.15.212.65:42413/ using document root <none> and password file <none>
I20260928 23:08:18.634539 27493 fs_manager.cc:359] Metadata directory not provided
I20260928 23:08:18.634618 27493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:08:18.634780 27493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:18.648950 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "344b240a463a425e8102041b7cb1173e"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "f7478f5463eb8455c7cf98eefe399e3a"
server_key_iv: "e060ff7b2354b411fad70122664330dc"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.649437 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "344b240a463a425e8102041b7cb1173e"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "f7478f5463eb8455c7cf98eefe399e3a"
server_key_iv: "e060ff7b2354b411fad70122664330dc"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.651252 27493 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260928 23:08:18.664211 27508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.664410 27493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:08:18.664518 27493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "344b240a463a425e8102041b7cb1173e"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "f7478f5463eb8455c7cf98eefe399e3a"
server_key_iv: "e060ff7b2354b411fad70122664330dc"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.664606 27493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:08:18.677789 27493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:08:18.679724 27493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:08:18.680243 27493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:08:18.680794 27493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:08:18.681172 27493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:08:18.681227 27493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.681291 27493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:08:18.681325 27493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.688066 27493 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:34449
I20260928 23:08:18.688613 27493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:08:18.689613 27623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:34449 every 8 connection(s)
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:08:18.696296 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 27493
I20260928 23:08:18.696401 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:08:18.696616 16209 external_mini_cluster.cc:1428] Setting key dd6da57e49c1ae7fede5b2c4d413b410
I20260928 23:08:18.699514 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:18.689437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39031 (local address 127.15.212.65:60273)
0928 23:08:18.689996 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:18.690028 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:18.690492 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:18.690849 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:18.690854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:18.691041 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:08:18.691236 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:18.691239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.692001 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.692002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:18.692881 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:18.692883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.693324 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.693326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:18.693489 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:18.694069 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:08:18.694078 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:08:18.695707 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:08:18.697944 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:18.697950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:18.697952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:18.698235 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:18.698452 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:18.698453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:18.698453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:18.698580 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:18.698868 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:08:18.698873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:08:18.699032 (+ 159us) client_negotiation.cc:769] Sending connection context
0928 23:08:18.699149 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 23:08:18.699292 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":67,"threads_started":1}
I20260928 23:08:18.700227 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:18.689711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:60273 (local address 127.15.212.126:39031)
0928 23:08:18.690646 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:18.690648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:18.690654 (+ 6us) server_negotiation.cc:407] Connection header received
0928 23:08:18.690676 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:18.690676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:18.690686 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:18.690773 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:08:18.691339 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.691883 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.692978 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.693207 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.693313 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:18.695833 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:08:18.695834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:08:18.695835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:08:18.695842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:08:18.697852 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:18.698322 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:18.698323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:18.698323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:18.698385 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:18.698654 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:18.698655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:18.698656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:18.698786 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:08:18.698843 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:18.700031 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0928 23:08:18.700129 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":109,"threads_started":1}
I20260928 23:08:18.700815 27624 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39031
I20260928 23:08:18.700960 27624 heartbeater.cc:464] Registering TS with master...
I20260928 23:08:18.701205 27624 heartbeater.cc:511] Master 127.15.212.126:39031 requested a full tablet report, sending...
I20260928 23:08:18.701778 27412 ts_manager.cc:194] Registered new tserver with Master: 344b240a463a425e8102041b7cb1173e (127.15.212.65:34449)
I20260928 23:08:18.702728 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:60273
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:08:18.730604 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:39091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:08:18.837987 27630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:08:18.838207 27630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:08:18.838244 27630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:08:18.839403 27630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:08:18.839470 27630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:08:18.839514 27630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:08:18.841046 27630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.212.84:39091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:08:18.841305 27630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:08:18.841512 27630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:08:18.846447 27638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:18.846447 27637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:18.846989 27640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:08:18.847405 27630 server_base.cc:1034] running on GCE node
I20260928 23:08:18.847638 27630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:08:18.847904 27630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:08:18.849045 27630 hybrid_clock.cc:648] HybridClock initialized: now 1790636898849027 us; error 36 us; skew 500 ppm
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:18.851538 27630 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:08:18.851938 27630 webserver.cc:460] Webserver started at http://127.15.212.66:33107/ using document root <none> and password file <none>
I20260928 23:08:18.852164 27630 fs_manager.cc:359] Metadata directory not provided
I20260928 23:08:18.852234 27630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:08:18.852378 27630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:18.864830 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "521a996586ac4b21860d1e765d0eca8f"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "dd90cdee4a4962c08f2ad5e9ba5b7e1e"
server_key_iv: "c90ead6107aa65a785ca9f77d8c5a038"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.865309 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "521a996586ac4b21860d1e765d0eca8f"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "dd90cdee4a4962c08f2ad5e9ba5b7e1e"
server_key_iv: "c90ead6107aa65a785ca9f77d8c5a038"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.870576 27630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:18.896683 27647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.896932 27630 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260928 23:08:18.897018 27630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "521a996586ac4b21860d1e765d0eca8f"
format_stamp: "Formatted at 2026-09-28 23:08:18 on dist-test-slave-x2t8"
server_key: "dd90cdee4a4962c08f2ad5e9ba5b7e1e"
server_key_iv: "c90ead6107aa65a785ca9f77d8c5a038"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:18.897105 27630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:08:18.949088 27630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:08:18.951632 27630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:08:18.951833 27630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:08:18.952045 27630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:08:18.952375 27630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:08:18.952416 27630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.952469 27630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:08:18.952528 27630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:18.960259 27630 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:45709
I20260928 23:08:18.960829 27630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:08:18.961306 27761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:45709 every 8 connection(s)
Sep 28 23:08:18 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:08:18.970865 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 27630
I20260928 23:08:18.970988 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:08:18.971184 16209 external_mini_cluster.cc:1428] Setting key f7bae7c4606348eaa500ffc390715434
I20260928 23:08:18.972054 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:18.961842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:58771 (local address 127.15.212.126:39031)
0928 23:08:18.961886 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:18.961888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:18.961898 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:08:18.962410 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:18.962412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:18.962426 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:18.962470 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:08:18.963183 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.963973 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.965488 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.965800 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.965845 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:18.968515 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:08:18.968516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:08:18.968517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:08:18.968524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:08:18.970290 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:18.970808 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:18.970809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:18.970810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:18.970869 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:18.971298 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:18.971299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:18.971300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:18.971448 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:08:18.971491 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:18.971870 (+ 379us) server_negotiation.cc:299] Negotiation successful
0928 23:08:18.971959 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:08:18.972783 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:18.961536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39031 (local address 127.15.212.66:58771)
0928 23:08:18.961709 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:18.961751 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:18.962286 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:18.962537 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:18.962541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:18.962788 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:08:18.963058 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:18.963065 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.964095 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.964096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:18.965381 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:18.965384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:18.965952 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:18.965953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:18.966097 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:18.966690 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:08:18.966701 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:08:18.968369 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:08:18.970387 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:18.970393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:18.970395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:18.970715 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:18.970986 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:18.970987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:18.970987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:18.971162 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:18.971538 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:08:18.971544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:08:18.971709 (+ 165us) client_negotiation.cc:769] Sending connection context
0928 23:08:18.972365 (+ 656us) client_negotiation.cc:240] Negotiation successful
0928 23:08:18.972542 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260928 23:08:18.973522 27762 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39031
I20260928 23:08:18.973673 27762 heartbeater.cc:464] Registering TS with master...
I20260928 23:08:18.973907 27762 heartbeater.cc:511] Master 127.15.212.126:39031 requested a full tablet report, sending...
I20260928 23:08:18.974298 27412 ts_manager.cc:194] Registered new tserver with Master: 521a996586ac4b21860d1e765d0eca8f (127.15.212.66:45709)
I20260928 23:08:18.974788 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:58771
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:08:19.003230 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:39091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:08:19.122952 27767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:08:19.123266 27767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:08:19.123342 27767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:08:19.124933 27767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:08:19.125048 27767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:08:19.125124 27767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:08:19.127650 27767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.212.84:39091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:08:19.128083 27767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:08:19.128360 27767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:08:19.134464 27775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:19.134706 27773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:08:19.134531 27772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:08:19.135707 27767 server_base.cc:1034] running on GCE node
I20260928 23:08:19.136001 27767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:08:19.136229 27767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:08:19.137391 27767 hybrid_clock.cc:648] HybridClock initialized: now 1790636899137377 us; error 43 us; skew 500 ppm
Sep 28 23:08:19 dist-test-slave-x2t8 krb5kdc[26936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636899, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:19.140409 27767 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:08:19.140858 27767 webserver.cc:460] Webserver started at http://127.15.212.67:42783/ using document root <none> and password file <none>
I20260928 23:08:19.141085 27767 fs_manager.cc:359] Metadata directory not provided
I20260928 23:08:19.141129 27767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:08:19.141237 27767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:08:19 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636899, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:19.158394 27767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "12ecc19ae7574973ab0cc9692e25fbca"
format_stamp: "Formatted at 2026-09-28 23:08:19 on dist-test-slave-x2t8"
server_key: "17163dae49e32dd843ae087ca3783328"
server_key_iv: "12900ba3a3a805e0411d7e035e1620b2"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:19.158823 27767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "12ecc19ae7574973ab0cc9692e25fbca"
format_stamp: "Formatted at 2026-09-28 23:08:19 on dist-test-slave-x2t8"
server_key: "17163dae49e32dd843ae087ca3783328"
server_key_iv: "12900ba3a3a805e0411d7e035e1620b2"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:19.161093 27767 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260928 23:08:19.176314 27782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:19.176638 27767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:08:19.176762 27767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "12ecc19ae7574973ab0cc9692e25fbca"
format_stamp: "Formatted at 2026-09-28 23:08:19 on dist-test-slave-x2t8"
server_key: "17163dae49e32dd843ae087ca3783328"
server_key_iv: "12900ba3a3a805e0411d7e035e1620b2"
server_key_version: "kuduclusterkey@0"
I20260928 23:08:19.176869 27767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:08:19.204442 27767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:08:19.207041 27767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:08:19.207234 27767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:08:19.207563 27767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:08:19.208001 27767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:08:19.208092 27767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:19.208166 27767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:08:19.208223 27767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:08:19.215179 27767 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:46697
I20260928 23:08:19.215668 27767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:08:19.219732 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 27767
I20260928 23:08:19.219847 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:08:19.220041 16209 external_mini_cluster.cc:1428] Setting key 3d3c178463c907f26984225689521902
I20260928 23:08:19.219547 27895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:46697 every 8 connection(s)
Sep 28 23:08:19 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636899, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:08:19.233475 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.220597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:57571 (local address 127.15.212.126:39031)
0928 23:08:19.220635 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.220637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.222658 (+ 2021us) server_negotiation.cc:407] Connection header received
0928 23:08:19.223328 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.223329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.223341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.223373 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:08:19.224235 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.225003 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.226500 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.226793 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.226838 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.229508 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:08:19.229509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:08:19.229510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:08:19.229517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:08:19.231271 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:19.232308 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:19.232309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:19.232309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:19.232376 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:19.232731 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:19.232732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:19.232732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:19.232862 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:08:19.232906 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.233278 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.233364 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:08:19.234194 27898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.222014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:39031 (local address 127.15.212.67:57571)
0928 23:08:19.222536 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.222581 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.223203 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.223574 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.223578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.223815 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:08:19.224093 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.224100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.225118 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.225119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.226381 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.226384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.226946 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.226947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.227081 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.227676 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:08:19.227685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:08:19.229387 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:08:19.231370 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:19.231628 (+ 258us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:19.231631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:19.232211 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:19.232511 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:08:19.232511 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:08:19.232512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:08:19.232646 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:08:19.232956 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:08:19.232962 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:08:19.233117 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.233809 (+ 692us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.233965 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":77,"threads_started":1}
I20260928 23:08:19.234817 27896 heartbeater.cc:347] Connected to a master server at 127.15.212.126:39031
I20260928 23:08:19.234905 27896 heartbeater.cc:464] Registering TS with master...
I20260928 23:08:19.235080 27896 heartbeater.cc:511] Master 127.15.212.126:39031 requested a full tablet report, sending...
I20260928 23:08:19.235474 27412 ts_manager.cc:194] Registered new tserver with Master: 12ecc19ae7574973ab0cc9692e25fbca (127.15.212.67:46697)
I20260928 23:08:19.235954 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:57571
I20260928 23:08:19.242588 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:08:19.260637 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.244330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48922 (local address 127.15.212.126:39031)
0928 23:08:19.244369 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.244370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.252692 (+ 8322us) server_negotiation.cc:407] Connection header received
0928 23:08:19.252815 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.252817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.252827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.252858 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:08:19.253749 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.254505 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.255546 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.255852 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.255899 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.257598 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:08:19.257599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:08:19.257599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:08:19.257606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:08:19.259315 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:19.259719 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:19.259720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:19.259721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:19.259778 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:08:19.260010 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:08:19.260011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:08:19.260011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:08:19.260137 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:08:19.260209 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.260428 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.260539 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:08:19.262921 27412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:08:19.265192 27412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:08:19.274526 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.269539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:34449 (local address 127.0.0.1:39652)
0928 23:08:19.270236 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.270244 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.270281 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.270513 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.270515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.270517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:08:19.270581 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.270583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.271854 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.271855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.273119 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.273121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.274246 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.274247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.274334 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.274346 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.274404 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.274450 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":72,"threads_started":1}
I20260928 23:08:19.274827 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.269780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.15.212.65:34449)
0928 23:08:19.270219 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.270224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.270327 (+ 103us) server_negotiation.cc:407] Connection header received
0928 23:08:19.270359 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.270361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.270377 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.270438 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:08:19.270648 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.271761 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.273214 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.273947 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.274027 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.274084 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.274713 (+ 629us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.274756 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":69,"threads_started":1}
I20260928 23:08:19.275491 27558 tablet_service.cc:1467] Processing CreateTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 (DEFAULT_TABLE table=test-table [id=2ff735bb66484fc0bd18b4edb69943ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:08:19.275892 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a23d30ef5e944eaea3ad41b8a7e87b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:08:19.282634 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.269762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:45709 (local address 127.0.0.1:40348)
0928 23:08:19.274137 (+ 4375us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.274147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.274189 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.275538 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.275539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.275541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:08:19.275576 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.275578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.280216 (+ 4638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.280217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.281355 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.281357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.282408 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.282409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.282468 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.282473 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.282520 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.282558 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4359,"thread_start_us":71,"threads_started":1}
I20260928 23:08:19.283212 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.269544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:46697 (local address 127.0.0.1:35934)
0928 23:08:19.270004 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.270043 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.270108 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.275670 (+ 5562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.275671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.275672 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:08:19.275709 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.275710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.278194 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.278195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.279288 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.279290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.283027 (+ 3737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.283028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.283087 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.283090 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.283117 (+ 27us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.283148 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":81,"threads_started":1}
I20260928 23:08:19.283730 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.270065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.15.212.67:46697)
0928 23:08:19.275121 (+ 5056us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.275125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.275135 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:08:19.275171 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.275173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.275187 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.275793 (+ 606us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:08:19.275863 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.278095 (+ 2232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.279402 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.280113 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.283508 (+ 3395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.283591 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.283616 (+ 25us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.283655 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5020,"thread_start_us":80,"threads_started":1}
I20260928 23:08:19.284044 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.270255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.15.212.66:45709)
0928 23:08:19.275329 (+ 5074us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.275333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.275341 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:08:19.275374 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.275375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.275390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.275459 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:08:19.276698 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.277757 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.281468 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.282244 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.282310 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.282376 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.283913 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.283953 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5041,"thread_start_us":76,"threads_started":1}
I20260928 23:08:19.284783 27818 tablet_service.cc:1467] Processing CreateTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 (DEFAULT_TABLE table=test-table [id=2ff735bb66484fc0bd18b4edb69943ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:08:19.284852 27678 tablet_service.cc:1467] Processing CreateTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 (DEFAULT_TABLE table=test-table [id=2ff735bb66484fc0bd18b4edb69943ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:08:19.285122 27818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a23d30ef5e944eaea3ad41b8a7e87b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:08:19.285122 27678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a23d30ef5e944eaea3ad41b8a7e87b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:08:19.288594 27916 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Bootstrap starting.
I20260928 23:08:19.289613 27916 tablet_bootstrap.cc:654] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Neither blocks nor log segments found. Creating new log.
I20260928 23:08:19.289953 27916 log.cc:824] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:08:19.290688 27916 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: No bootstrap required, opened a new log
I20260928 23:08:19.290747 27916 ts_tablet_manager.cc:1397] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:08:19.292527 27916 raft_consensus.cc:348] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.292686 27916 raft_consensus.cc:374] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:08:19.292723 27916 raft_consensus.cc:729] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521a996586ac4b21860d1e765d0eca8f, State: Initialized, Role: FOLLOWER
I20260928 23:08:19.292821 27915 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Bootstrap starting.
I20260928 23:08:19.292830 27916 consensus_queue.cc:260] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.293078 27916 ts_tablet_manager.cc:1428] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:08:19.293275 27762 heartbeater.cc:503] Master 127.15.212.126:39031 was elected leader, sending a full tablet report...
I20260928 23:08:19.293525 27915 tablet_bootstrap.cc:654] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Neither blocks nor log segments found. Creating new log.
I20260928 23:08:19.293807 27915 log.cc:824] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:08:19.296643 27917 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Bootstrap starting.
I20260928 23:08:19.298029 27917 tablet_bootstrap.cc:654] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Neither blocks nor log segments found. Creating new log.
I20260928 23:08:19.298497 27917 log.cc:824] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Log is configured to *not* fsync() on all Append() calls
I20260928 23:08:19.298568 27915 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: No bootstrap required, opened a new log
I20260928 23:08:19.298640 27915 ts_tablet_manager.cc:1397] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260928 23:08:19.300666 27915 raft_consensus.cc:348] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.300837 27915 raft_consensus.cc:374] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:08:19.300874 27915 raft_consensus.cc:729] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 344b240a463a425e8102041b7cb1173e, State: Initialized, Role: FOLLOWER
I20260928 23:08:19.301007 27915 consensus_queue.cc:260] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.301249 27915 ts_tablet_manager.cc:1428] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:08:19.301390 27624 heartbeater.cc:503] Master 127.15.212.126:39031 was elected leader, sending a full tablet report...
I20260928 23:08:19.314888 27917 tablet_bootstrap.cc:492] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: No bootstrap required, opened a new log
I20260928 23:08:19.315080 27917 ts_tablet_manager.cc:1397] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260928 23:08:19.317234 27917 raft_consensus.cc:348] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.318776 27917 raft_consensus.cc:374] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:08:19.318867 27917 raft_consensus.cc:729] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ecc19ae7574973ab0cc9692e25fbca, State: Initialized, Role: FOLLOWER
I20260928 23:08:19.319034 27917 consensus_queue.cc:260] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.319360 27917 ts_tablet_manager.cc:1428] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260928 23:08:19.319476 27896 heartbeater.cc:503] Master 127.15.212.126:39031 was elected leader, sending a full tablet report...
Sep 28 23:08:19 dist-test-slave-x2t8 krb5kdc[26936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636898, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
W20260928 23:08:19.440392 27625 tablet.cc:2369] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:08:19.461961 27763 tablet.cc:2369] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:08:19.472595 27897 tablet.cc:2369] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:08:19.522749 27922 raft_consensus.cc:482] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:08:19.522912 27922 raft_consensus.cc:504] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.523319 27922 leader_election.cc:288] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ecc19ae7574973ab0cc9692e25fbca (127.15.212.67:46697), 521a996586ac4b21860d1e765d0eca8f (127.15.212.66:45709)
I20260928 23:08:19.528286 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.524505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:51751 (local address 127.15.212.67:46697)
0928 23:08:19.524552 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.524554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.524563 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:08:19.524602 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.524602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.524617 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.524675 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:08:19.524918 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.525681 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.527330 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.527864 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.527916 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.527925 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.528162 (+ 237us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.528204 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 23:08:19.528779 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.523558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:46697 (local address 127.15.212.65:51751)
0928 23:08:19.523812 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.523824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.523885 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.524731 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.524733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.524747 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:08:19.524831 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.524832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.525805 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.525806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.527229 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.527232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.527986 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.527987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.528056 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.528059 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.528634 (+ 575us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.528677 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":100,"threads_started":1}
I20260928 23:08:19.529318 27838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a23d30ef5e944eaea3ad41b8a7e87b20" candidate_uuid: "344b240a463a425e8102041b7cb1173e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ecc19ae7574973ab0cc9692e25fbca" is_pre_election: true
I20260928 23:08:19.529491 27838 raft_consensus.cc:2454] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344b240a463a425e8102041b7cb1173e in term 0.
I20260928 23:08:19.529734 27509 leader_election.cc:302] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ecc19ae7574973ab0cc9692e25fbca, 344b240a463a425e8102041b7cb1173e; no voters:
I20260928 23:08:19.529883 27922 raft_consensus.cc:2781] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:08:19.529942 27922 raft_consensus.cc:482] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:08:19.529976 27922 raft_consensus.cc:3037] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:08:19.530893 27922 raft_consensus.cc:504] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.531030 27922 leader_election.cc:288] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [CANDIDATE]: Term 1 election: Requested vote from peers 12ecc19ae7574973ab0cc9692e25fbca (127.15.212.67:46697), 521a996586ac4b21860d1e765d0eca8f (127.15.212.66:45709)
I20260928 23:08:19.531265 27838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a23d30ef5e944eaea3ad41b8a7e87b20" candidate_uuid: "344b240a463a425e8102041b7cb1173e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ecc19ae7574973ab0cc9692e25fbca"
I20260928 23:08:19.531353 27838 raft_consensus.cc:3037] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:08:19.531401 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.526409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:34973 (local address 127.15.212.66:45709)
0928 23:08:19.526458 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.526460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.526470 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:08:19.526560 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.526561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.526570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.526600 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:08:19.526810 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.527527 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.528939 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.529473 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.529523 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.529531 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.531293 (+ 1762us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.531337 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 23:08:19.531643 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.526207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:45709 (local address 127.15.212.65:34973)
0928 23:08:19.526415 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.526425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:08:19.526475 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:08:19.526667 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:08:19.526668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:08:19.526670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:08:19.526738 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.526739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.527624 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.527625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.528841 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:08:19.528845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.529633 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.529634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:08:19.531185 (+ 1551us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.531189 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:08:19.531275 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 23:08:19.531313 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260928 23:08:19.531960 27716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a23d30ef5e944eaea3ad41b8a7e87b20" candidate_uuid: "344b240a463a425e8102041b7cb1173e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521a996586ac4b21860d1e765d0eca8f" is_pre_election: true
I20260928 23:08:19.532132 27716 raft_consensus.cc:2454] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 344b240a463a425e8102041b7cb1173e in term 0.
I20260928 23:08:19.531965 27715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a23d30ef5e944eaea3ad41b8a7e87b20" candidate_uuid: "344b240a463a425e8102041b7cb1173e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521a996586ac4b21860d1e765d0eca8f"
I20260928 23:08:19.532291 27838 raft_consensus.cc:2454] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344b240a463a425e8102041b7cb1173e in term 1.
I20260928 23:08:19.532310 27715 raft_consensus.cc:3037] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:08:19.532426 27509 leader_election.cc:302] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ecc19ae7574973ab0cc9692e25fbca, 344b240a463a425e8102041b7cb1173e; no voters:
I20260928 23:08:19.532657 27922 raft_consensus.cc:2781] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:08:19.532792 27922 raft_consensus.cc:686] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 1 LEADER]: Becoming Leader. State: Replica: 344b240a463a425e8102041b7cb1173e, State: Running, Role: LEADER
I20260928 23:08:19.532871 27922 consensus_queue.cc:237] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } }
I20260928 23:08:19.533346 27715 raft_consensus.cc:2454] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 344b240a463a425e8102041b7cb1173e in term 1.
I20260928 23:08:19.533664 27412 catalog_manager.cc:5495] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e reported cstate change: term changed from 0 to 1, leader changed from <none> to 344b240a463a425e8102041b7cb1173e (127.15.212.65). New cstate: current_term: 1 leader_uuid: "344b240a463a425e8102041b7cb1173e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "344b240a463a425e8102041b7cb1173e" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 34449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 } health_report { overall_health: UNKNOWN } } }
I20260928 23:08:19.555426 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.552255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.15.212.65:34449)
0928 23:08:19.552314 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 23:08:19.552316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:08:19.552324 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:08:19.552357 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:08:19.552358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:08:19.552377 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:08:19.552471 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:08:19.552639 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.553334 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.554440 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:08:19.554739 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:08:19.554807 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:08:19.555006 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:08:19.555124 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:08:19.555206 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:08:19.555278 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 23:08:19.555306 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 23:08:19.557745 27715 raft_consensus.cc:1261] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 1 FOLLOWER]: Refusing update from remote peer 344b240a463a425e8102041b7cb1173e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:08:19.557749 27838 raft_consensus.cc:1261] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 1 FOLLOWER]: Refusing update from remote peer 344b240a463a425e8102041b7cb1173e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:08:19.558053 27926 consensus_queue.cc:1035] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ecc19ae7574973ab0cc9692e25fbca" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 46697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:08:19.558149 27922 consensus_queue.cc:1035] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [LEADER]: Connected to new peer: Peer: permanent_uuid: "521a996586ac4b21860d1e765d0eca8f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 45709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:08:19.561123 27931 mvcc.cc:205] Tried to move back new op lower bound from 7334448740585418752 to 7334448740487118848. Current Snapshot: MvccSnapshot[applied={T|T < 7334448740585418752}]
I20260928 23:08:19.561709 27929 mvcc.cc:205] Tried to move back new op lower bound from 7334448740585418752 to 7334448740487118848. Current Snapshot: MvccSnapshot[applied={T|T < 7334448740585418752 or (T in {7334448740585418752})}]
I20260928 23:08:19.563184 27412 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:08:19.563280 27412 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:08:19.563988 27412 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c927a5ed02e452eb0a4a46cef2535d8: Sending DeleteTablet for 3 replicas of tablet a23d30ef5e944eaea3ad41b8a7e87b20
I20260928 23:08:19.564352 27930 mvcc.cc:205] Tried to move back new op lower bound from 7334448740585418752 to 7334448740487118848. Current Snapshot: MvccSnapshot[applied={T|T < 7334448740585418752}]
I20260928 23:08:19.564446 27818 tablet_service.cc:1514] Processing DeleteTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:08:19 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:35934
I20260928 23:08:19.564677 27678 tablet_service.cc:1514] Processing DeleteTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:08:19 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:40348
I20260928 23:08:19.565016 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet a23d30ef5e944eaea3ad41b8a7e87b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:08:19 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:39652
I20260928 23:08:19.565235 27940 tablet_replica.cc:330] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e: stopping tablet replica
I20260928 23:08:19.565369 27940 raft_consensus.cc:2229] T a23d30ef5e944eaea3ad41b8a7e87b20 P 344b240a463a425e8102041b7cb1173e [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:08:19.565583 27938 tablet_replica.cc:330] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: stopping tablet replica
I20260928 23:08:19.565672 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 27493
I20260928 23:08:19.565685 27938 raft_consensus.cc:2229] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:08:19.565764 27938 raft_consensus.cc:2258] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:08:19.568236 27939 tablet_replica.cc:330] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: stopping tablet replica
I20260928 23:08:19.568337 27939 raft_consensus.cc:2229] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:08:19.568424 27939 raft_consensus.cc:2258] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:08:19.568872 27939 ts_tablet_manager.cc:1905] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:08:19.568981 27938 ts_tablet_manager.cc:1905] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:08:19.570698 27939 ts_tablet_manager.cc:1918] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:08:19.570779 27939 log.cc:1196] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/a23d30ef5e944eaea3ad41b8a7e87b20
I20260928 23:08:19.571033 27939 ts_tablet_manager.cc:1939] T a23d30ef5e944eaea3ad41b8a7e87b20 P 521a996586ac4b21860d1e765d0eca8f: Deleting consensus metadata
I20260928 23:08:19.571280 27938 ts_tablet_manager.cc:1918] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:08:19.571396 27938 log.cc:1196] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/a23d30ef5e944eaea3ad41b8a7e87b20
I20260928 23:08:19.571411 27399 catalog_manager.cc:4841] TS 521a996586ac4b21860d1e765d0eca8f (127.15.212.66:45709): tablet a23d30ef5e944eaea3ad41b8a7e87b20 (table test-table [id=2ff735bb66484fc0bd18b4edb69943ae]) successfully deleted
I20260928 23:08:19.572016 27938 ts_tablet_manager.cc:1939] T a23d30ef5e944eaea3ad41b8a7e87b20 P 12ecc19ae7574973ab0cc9692e25fbca: Deleting consensus metadata
I20260928 23:08:19.572424 27396 catalog_manager.cc:4841] TS 12ecc19ae7574973ab0cc9692e25fbca (127.15.212.67:46697): tablet a23d30ef5e944eaea3ad41b8a7e87b20 (table test-table [id=2ff735bb66484fc0bd18b4edb69943ae]) successfully deleted
W20260928 23:08:19.576522 27397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:34449 (error 108)
I20260928 23:08:19.576679 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 27630
I20260928 23:08:19.577819 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:08:19.577487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:34449 (local address 127.0.0.1:39672)
0928 23:08:19.577738 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0928 23:08:19.577779 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:34449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
W20260928 23:08:19.578480 27397 catalog_manager.cc:4584] TS 344b240a463a425e8102041b7cb1173e (127.15.212.65:34449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a23d30ef5e944eaea3ad41b8a7e87b20: Network error: Client connection negotiation failed: client connection to 127.15.212.65:34449: connect: Connection refused (error 111)
I20260928 23:08:19.581964 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 27767
I20260928 23:08:19.588104 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 27380
I20260928 23:08:19.593447 16209 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 23:08:19.927348 16209 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 23:08:19.927438 16209 mini_ranger.cc:66] Stopping Ranger...
I20260928 23:08:20.272152 16209 mini_ranger.cc:68] Stopped Ranger
2026-09-28 23:08:20.272 UTC [26966] LOG: received smart shutdown request
2026-09-28 23:08:20.276 UTC [26966] LOG: background worker "logical replication launcher" (PID 26975) exited with exit code 1
2026-09-28 23:08:20.276 UTC [26970] LOG: shutting down
2026-09-28 23:08:20.424 UTC [26966] LOG: database system is shut down
2026-09-28T23:08:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45365 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:08:20 dist-test-slave-x2t8 krb5kdc[27950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:08:20 dist-test-slave-x2t8 krb5kdc[27950](info): set up 2 sockets
Sep 28 23:08:20 dist-test-slave-x2t8 krb5kdc[27950](info): commencing operation
krb5kdc: starting...
W20260928 23:08:21.556284 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:08:21 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:08:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangeradmin_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangeradmin_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangerlookup_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangerlookup_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
I20260928 23:08:21.708925 16209 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskdR8rxt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/postgres -l logfile start
2026-09-28 23:08:24.135 UTC [27980] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 23:08:24.135 UTC [27980] LOG: listening on IPv4 address "127.15.212.84", port 47599
2026-09-28 23:08:24.139 UTC [27980] LOG: listening on Unix socket "/tmp/.s.PGSQL.47599"
2026-09-28 23:08:24.160 UTC [27983] LOG: database system was shut down at 2026-09-28 23:08:22 UTC
2026-09-28 23:08:24.163 UTC [27980] LOG: database system is ready to accept connections
I20260928 23:08:25.116166 16209 mini_postgres.cc:96] Postgres bound to 47599
2026-09-28 23:08:25.122 UTC [27991] FATAL: database "slave" does not exist
127.15.212.84:47599 - accepting connections
I20260928 23:08:25.122622 16209 mini_ranger.cc:161] Starting Ranger...
I20260928 23:08:25.136896 16209 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 23:08:25.592633 16209 mini_ranger.cc:87] Created ranger Postgres database
I20260928 23:08:25.592723 16209 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin
2026-09-28 23:08:25,861 [I] DB FLAVOR :POSTGRES
2026-09-28 23:08:25,862 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 23:08:25,862 [I] Checking connection..
2026-09-28 23:08:25,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 23:08:26,112 [I] Checking connection passed.
2026-09-28 23:08:26,112 [I] --------- Verifying version history table ---------
2026-09-28 23:08:26,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:08:26,368 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 23:08:26,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:08:26,617 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 23:08:26,617 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 23:08:26,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 23:08:26.866 UTC [28055] WARNING: there is no transaction in progress
2026-09-28 23:08:26,873 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 23:08:26,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 23:08:27,092 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 23:08:27,093 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 23:08:27,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 23:08:27,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 23:08:27,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x2t8.c.gcp-upstream.internal','N') ;"
2026-09-28 23:08:27,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 23:08:28,000 [I] Table x_portal_user does not exist in database ranger
2026-09-28 23:08:28,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 23:08:28,229 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 23:08:28,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 23:08:28,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 23:08:28,678 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 23:08:28,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 23:08:29.540 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.555 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.570 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.584 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.599 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.618 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.668 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.674 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.687 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.696 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.705 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.718 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.726 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.732 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.739 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:29.745 UTC [28163] WARNING: there is no transaction in progress
2026-09-28 23:08:30,333 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 23:08:30,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 23:08:30,579 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 23:08:30,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 23:08:30,829 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 23:08:30,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 23:08:31,052 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 23:08:31,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 23:08:31,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 23:08:31,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x2t8.c.gcp-upstream.internal';"
2026-09-28 23:08:31,758 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 23:08:31,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 23:08:31,997 [I] DB_PATCHES have already been applied
I20260928 23:08:32.002681 16209 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskdR8rxt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 23:08:32.002760 16209 mini_ranger.cc:193] Using host: 127.15.212.84
I20260928 23:08:32.005721 16209 mini_ranger.cc:227] Ranger admin URL: http://127.15.212.84:46509
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 11:08:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 23:08:33.089869 16209 mini_ranger.cc:160] Time spent starting Ranger: real 7.967s user 0.002s sys 0.006s
Sep 28, 2026 11:08:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 11:08:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 11:08:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 11:08:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 11:08:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 23:08:54.407560 16209 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.212.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.212.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangerkms_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.212.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/rangerkms_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.212.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.212.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.212.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/HTTP_127.15.212.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 23:08:54 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636934, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 23:08:54.492221 16209 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 23:08:54.501173 16209 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 23:08:55.371055 16209 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 23:08:55.371140 16209 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms
I20260928 23:08:55.371156 16209 mini_ranger_kms.cc:227] Using postgres at 127.15.212.84:47599
2026-09-28 23:08:55,553 [I] DB FLAVOR :POSTGRES
2026-09-28 23:08:55,553 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 23:08:55,553 [I] Checking connection
2026-09-28 23:08:55,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 23:08:55,810 [I] connection success
2026-09-28 23:08:55,811 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 23:08:55,811 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 23:08:55,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 23:08:56,083 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 23:08:56,083 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 23:08:56,083 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 23:08:56,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.212.84:47599/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 23:08:56,353 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 23:08:56.812507 16209 mini_ranger_kms.cc:310] Created kms service
I20260928 23:08:56.995596 16209 mini_ranger_kms.cc:326] Created kudu user
I20260928 23:08:57.040831 16209 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 23:08:57 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636934, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:08:57.416301 16209 mini_ranger_kms.cc:384] Added ranger policy
I20260928 23:08:57.416633 16209 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms:/tmp/dist-test-taskdR8rxt/build/release/bin/postgresql.jar:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskdR8rxt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 23:08:57.416671 16209 mini_ranger_kms.cc:240] Using host: 127.15.212.84
I20260928 23:08:57.419946 16209 mini_ranger_kms.cc:276] Ranger KMS PID: 28343
I20260928 23:08:57.420020 16209 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.212.84:47809
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 11:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 28 23:08:59 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790636939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:08:59.691890 16209 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.200s user 0.027s sys 0.138s
I20260928 23:08:59.692008 16209 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 23:08:59.692054 16209 mini_ranger_kms.cc:401] 127.15.212.84:47809/kms/v1/keys
Sep 28 23:09:00 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
Sep 28 23:09:00 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.212.84@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
Sep 28 23:09:02 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:09:02.673473 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34113
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34113
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:02.752206 28394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:02.752390 28394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:02.752413 28394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:02.753546 28394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:02.753587 28394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:02.753602 28394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:02.753613 28394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:02.753625 28394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:02.755020 28394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.212.84:47809/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34113
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:02.755191 28394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:02.755358 28394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:02.759914 28400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:02.759924 28399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:02.759924 28402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:02.760347 28394 server_base.cc:1034] running on GCE node
I20260928 23:09:02.760618 28394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:02.760838 28394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:02.761988 28394 hybrid_clock.cc:648] HybridClock initialized: now 1790636942761968 us; error 40 us; skew 500 ppm
Sep 28 23:09:02 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:02.764252 28394 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:02.764722 28394 webserver.cc:460] Webserver started at http://127.15.212.126:46123/ using document root <none> and password file <none>
I20260928 23:09:02.764922 28394 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:02.764958 28394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:02.765096 28394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:09:02 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636942, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:02.817714 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "3a5441fc21754e74a96e2455296f1071"
format_stamp: "Formatted at 2026-09-28 23:09:02 on dist-test-slave-x2t8"
server_key: "e298557221c0dcc2bbecef006c05f3e0"
server_key_iv: "ef1028f53983a6885eff01909b2d9cc9"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:02.818153 28394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "3a5441fc21754e74a96e2455296f1071"
format_stamp: "Formatted at 2026-09-28 23:09:02 on dist-test-slave-x2t8"
server_key: "e298557221c0dcc2bbecef006c05f3e0"
server_key_iv: "ef1028f53983a6885eff01909b2d9cc9"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:02.819839 28394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:09:02.830358 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:02.830592 28394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:02.830698 28394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "3a5441fc21754e74a96e2455296f1071"
format_stamp: "Formatted at 2026-09-28 23:09:02 on dist-test-slave-x2t8"
server_key: "e298557221c0dcc2bbecef006c05f3e0"
server_key_iv: "ef1028f53983a6885eff01909b2d9cc9"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:02.830777 28394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:02.855484 28394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:02.857565 28394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:02.857717 28394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:02.862589 28394 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:34113
I20260928 23:09:02.862686 28461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:34113 every 8 connection(s)
I20260928 23:09:02.863024 28394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:02.863718 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:02.865949 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Bootstrap starting.
I20260928 23:09:02.866499 28462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:02.866777 28462 log.cc:824] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:02.867478 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: No bootstrap required, opened a new log
I20260928 23:09:02.868351 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28394
I20260928 23:09:02.868470 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:09:02.868644 16209 external_mini_cluster.cc:1428] Setting key c8b27f580beaf6e891c6c52a462fd9ca
I20260928 23:09:02.870216 28462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:02.870369 28462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:02.870429 28462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5441fc21754e74a96e2455296f1071, State: Initialized, Role: FOLLOWER
I20260928 23:09:02.870600 28462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:02.870707 28462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:02.870740 28462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:02.870817 28462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:02.871501 28462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:02.871618 28462 leader_election.cc:302] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a5441fc21754e74a96e2455296f1071; no voters:
I20260928 23:09:02.871819 28462 leader_election.cc:288] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:09:02.871845 28467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:02.872059 28467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 LEADER]: Becoming Leader. State: Replica: 3a5441fc21754e74a96e2455296f1071, State: Running, Role: LEADER
I20260928 23:09:02.872143 28462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:02.872212 28467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:02.872573 28467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a5441fc21754e74a96e2455296f1071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } } }
I20260928 23:09:02.872732 28467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:02.872906 28467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a5441fc21754e74a96e2455296f1071. Latest consensus state: current_term: 1 leader_uuid: "3a5441fc21754e74a96e2455296f1071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } } }
I20260928 23:09:02.873023 28467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:02.873196 28476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:02.873731 28476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:02.873960 28462 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 23:09:02.876695 28484 mvcc.cc:205] Tried to move back new op lower bound from 7334448918014427136 to 7334448918005088256. Current Snapshot: MvccSnapshot[applied={T|T < 7334448918014427136}]
Sep 28 23:09:02 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:02.882165 28476 catalog_manager.cc:1347] Generated new cluster ID: 69d1b97c1e744b3d97a047ae073242b6
I20260928 23:09:02.882246 28476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:02.882984 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:02.869161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.15.212.126:34113)
0928 23:09:02.869342 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:02.869351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:02.869383 (+ 32us) server_negotiation.cc:407] Connection header received
0928 23:09:02.870234 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:02.870244 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:02.870315 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:02.870474 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:02.871342 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:02.872787 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:02.873942 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:02.874563 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:02.877364 (+ 2801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:02.877774 (+ 410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:02.877778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:02.877779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:02.877795 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:02.880262 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:02.880694 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:02.880697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:02.880698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:02.880845 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:02.881097 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:02.881108 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:02.881108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:02.881587 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:02.881756 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:02.882284 (+ 528us) server_negotiation.cc:299] Negotiation successful
0928 23:09:02.882665 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260928 23:09:02.895947 28476 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:09:02.896730 28476 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:02.901958 28476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Generated new TSK 0
I20260928 23:09:02.902163 28476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:09:03.616164 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:03.709969 28507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:03.710201 28507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:03.710239 28507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:03.711449 28507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:03.711511 28507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:03.711553 28507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:03.713052 28507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:03.713306 28507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:03.713536 28507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:03.719334 28513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:03.719411 28512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:03.719357 28515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:03.719684 28507 server_base.cc:1034] running on GCE node
I20260928 23:09:03.719908 28507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:03.720165 28507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:03.721347 28507 hybrid_clock.cc:648] HybridClock initialized: now 1790636943721333 us; error 27 us; skew 500 ppm
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:03.724299 28507 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:03.724761 28507 webserver.cc:460] Webserver started at http://127.15.212.65:38841/ using document root <none> and password file <none>
I20260928 23:09:03.724996 28507 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:03.725062 28507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:03.725196 28507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:03.738121 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "cb55d9e741514a79abe8dc081c3b8a5f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "aa0719bb359550a2afaa84b0cf72652d"
server_key_iv: "b9fd9a3b01b700a3abb50f33d4c839c6"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.738536 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "cb55d9e741514a79abe8dc081c3b8a5f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "aa0719bb359550a2afaa84b0cf72652d"
server_key_iv: "b9fd9a3b01b700a3abb50f33d4c839c6"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.740278 28507 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260928 23:09:03.756630 28524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:03.756875 28507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 23:09:03.756994 28507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "cb55d9e741514a79abe8dc081c3b8a5f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "aa0719bb359550a2afaa84b0cf72652d"
server_key_iv: "b9fd9a3b01b700a3abb50f33d4c839c6"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.757084 28507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:03.782786 28507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:03.785281 28507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:03.785485 28507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:03.785753 28507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:03.786123 28507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:03.786203 28507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:03.786295 28507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:03.786370 28507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:03.793097 28507 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:33123
I20260928 23:09:03.793550 28507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:03.794646 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28507
I20260928 23:09:03.794734 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:09:03.794912 16209 external_mini_cluster.cc:1428] Setting key 802d33911fbf7a888580ae9ae5584f07
I20260928 23:09:03.793999 28637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:33123 every 8 connection(s)
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:03.807638 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:03.794596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.65:53633)
0928 23:09:03.795399 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:03.795444 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:03.796983 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:03.797319 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:03.797323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:03.797593 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:03.797858 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:03.797865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:03.798858 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:03.798859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:03.800020 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:03.800023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:03.800608 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:03.800609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:03.800748 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:03.801322 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:03.801332 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:03.802842 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:03.804775 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:03.804781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:03.804783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:03.805075 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:03.805281 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:03.805281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:03.805282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:03.805402 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:03.807049 (+ 1647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:03.807055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:03.807198 (+ 143us) client_negotiation.cc:769] Sending connection context
0928 23:09:03.807300 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 23:09:03.807430 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":91,"threads_started":1}
I20260928 23:09:03.808163 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:03.795163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:53633 (local address 127.15.212.126:34113)
0928 23:09:03.795335 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:03.795337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:03.796543 (+ 1206us) server_negotiation.cc:407] Connection header received
0928 23:09:03.797135 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:03.797136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:03.797149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:03.797211 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:03.797983 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:03.798753 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:03.800115 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:03.800436 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:03.800528 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:03.802937 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:03.802938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:03.802939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:03.802957 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:03.804688 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:03.805156 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:03.805157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:03.805158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:03.805217 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:03.805467 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:03.805468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:03.805468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:03.805607 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:03.805666 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:03.807999 (+ 2333us) server_negotiation.cc:299] Negotiation successful
0928 23:09:03.808078 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260928 23:09:03.808876 28638 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:03.808982 28638 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:03.809167 28638 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:03.809715 28426 ts_manager.cc:194] Registered new tserver with Master: cb55d9e741514a79abe8dc081c3b8a5f (127.15.212.65:33123)
I20260928 23:09:03.810565 28426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:53633
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:09:03.834852 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 23:09:03.954311 28644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:03.954641 28644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:03.954727 28644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:03.956429 28644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:03.956507 28644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:03.956543 28644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:03.958657 28644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:03.959087 28644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:03.959400 28644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:03.966189 28651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:03.966297 28652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:03.966822 28654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:03.967218 28644 server_base.cc:1034] running on GCE node
I20260928 23:09:03.967430 28644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:03.967667 28644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:03.968825 28644 hybrid_clock.cc:648] HybridClock initialized: now 1790636943968813 us; error 30 us; skew 500 ppm
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:03.971215 28644 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:03.971683 28644 webserver.cc:460] Webserver started at http://127.15.212.66:37825/ using document root <none> and password file <none>
I20260928 23:09:03.971977 28644 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:03.972049 28644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:03.972211 28644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:09:03 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:03.984175 28644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "67e1ea4c028bf61a524bbcaca461efc3"
server_key_iv: "4ddc597133c15d55cd140ee7e77305d0"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.984604 28644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "67e1ea4c028bf61a524bbcaca461efc3"
server_key_iv: "4ddc597133c15d55cd140ee7e77305d0"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.986591 28644 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260928 23:09:03.998956 28661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:03.999214 28644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:03.999315 28644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "67e1ea4c028bf61a524bbcaca461efc3"
server_key_iv: "4ddc597133c15d55cd140ee7e77305d0"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:03.999409 28644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:04.013890 28644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:04.016327 28644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:04.016551 28644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:04.016875 28644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:04.017288 28644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:04.017385 28644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.017482 28644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:04.017555 28644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.024150 28644 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:42271
I20260928 23:09:04.024600 28644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:04.033007 28775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:42271 every 8 connection(s)
I20260928 23:09:04.034739 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28644
I20260928 23:09:04.034824 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:09:04.035012 16209 external_mini_cluster.cc:1428] Setting key 4dcbc06628a1dc30786196868e4bc5e9
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636943, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:04.043771 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.033231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:55363 (local address 127.15.212.126:34113)
0928 23:09:04.033285 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:04.033287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:04.033659 (+ 372us) server_negotiation.cc:407] Connection header received
0928 23:09:04.034355 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:04.034356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:04.034371 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:04.034406 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:04.035210 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.036005 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.037525 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.037818 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.037865 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.040404 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:04.040405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:04.040406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:04.040413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:04.042179 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.042685 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.042686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.042686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.042744 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.043040 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.043041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.043041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.043169 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:04.043212 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:04.043586 (+ 374us) server_negotiation.cc:299] Negotiation successful
0928 23:09:04.043670 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 23:09:04.044236 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.033221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.66:55363)
0928 23:09:04.033527 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:04.033570 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:04.034226 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:04.034599 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:04.034603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:04.034828 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:04.035090 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.035096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.036119 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.036120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.037385 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.037388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.037967 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.037968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.038100 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.038690 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:04.038700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:04.040292 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:04.042275 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.042281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.042283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.042578 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.042819 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.042820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.042821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.042946 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.043259 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:04.043265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:04.043412 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 23:09:04.043855 (+ 443us) client_negotiation.cc:240] Negotiation successful
0928 23:09:04.044014 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":101,"threads_started":1}
I20260928 23:09:04.044795 28776 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:04.044883 28776 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.045077 28776 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:04.045493 28426 ts_manager.cc:194] Registered new tserver with Master: ac64b8c72ff541b8b13e88bfd6a6b37f (127.15.212.66:42271)
I20260928 23:09:04.045990 28426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:55363
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:09:04.075368 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:04.179634 28781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:04.179948 28781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:04.180040 28781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:04.181774 28781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:04.181905 28781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:04.181991 28781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:04.184065 28781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:04.184331 28781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:04.184638 28781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:04.191036 28787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.191087 28786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.191036 28789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.192256 28781 server_base.cc:1034] running on GCE node
I20260928 23:09:04.192490 28781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:04.192759 28781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:04.193918 28781 hybrid_clock.cc:648] HybridClock initialized: now 1790636944193902 us; error 32 us; skew 500 ppm
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:04.196699 28781 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:04.197172 28781 webserver.cc:460] Webserver started at http://127.15.212.67:44419/ using document root <none> and password file <none>
I20260928 23:09:04.197564 28781 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:04.197654 28781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:04.197813 28781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:04.209558 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "8656ec78e9d6430d94d4fa718a116b3c"
format_stamp: "Formatted at 2026-09-28 23:09:04 on dist-test-slave-x2t8"
server_key: "f3b458f87ab9fec8d4b315a7b18eac03"
server_key_iv: "b46393138bf64fee05e19fbded2c6174"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.210062 28781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "8656ec78e9d6430d94d4fa718a116b3c"
format_stamp: "Formatted at 2026-09-28 23:09:04 on dist-test-slave-x2t8"
server_key: "f3b458f87ab9fec8d4b315a7b18eac03"
server_key_iv: "b46393138bf64fee05e19fbded2c6174"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.216840 28781 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.001s
I20260928 23:09:04.228554 28796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.228742 28781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:04.228822 28781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "8656ec78e9d6430d94d4fa718a116b3c"
format_stamp: "Formatted at 2026-09-28 23:09:04 on dist-test-slave-x2t8"
server_key: "f3b458f87ab9fec8d4b315a7b18eac03"
server_key_iv: "b46393138bf64fee05e19fbded2c6174"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.228894 28781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:04.244604 28781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:04.246896 28781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:04.247076 28781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:04.247355 28781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:04.247833 28781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:04.247916 28781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.247987 28781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:04.248045 28781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.254665 28781 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:38087
I20260928 23:09:04.255025 28781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:09:04.256488 28909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:38087 every 8 connection(s)
I20260928 23:09:04.258271 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28781
I20260928 23:09:04.258395 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:09:04.258582 16209 external_mini_cluster.cc:1428] Setting key d99e72d25093d4e2fe993f8d9ba48629
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:04.266350 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.257145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:49841 (local address 127.15.212.126:34113)
0928 23:09:04.257189 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:04.257191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:04.257530 (+ 339us) server_negotiation.cc:407] Connection header received
0928 23:09:04.258048 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:04.258049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:04.258060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:04.258126 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:04.258822 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.259548 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.260573 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.260858 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.260930 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.263144 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:04.263145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:04.263146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:04.263152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:04.264822 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.265297 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.265298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.265298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.265353 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.265618 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.265619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.265620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.265743 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:04.265784 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:04.266158 (+ 374us) server_negotiation.cc:299] Negotiation successful
0928 23:09:04.266253 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:09:04.267045 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.257148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.67:49841)
0928 23:09:04.257365 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:04.257420 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:04.257937 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:04.258204 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:04.258207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:04.258436 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:04.258705 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.258712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.259637 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.259638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.260491 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.260493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.261007 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.261008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.261110 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.261654 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:04.261668 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:04.263039 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:04.264918 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.264923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.264925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.265206 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.265423 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.265423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.265423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.265542 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.265828 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:04.265833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:04.265994 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 23:09:04.266675 (+ 681us) client_negotiation.cc:240] Negotiation successful
0928 23:09:04.266823 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260928 23:09:04.267637 28910 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:04.267709 28910 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.267880 28910 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:04.268306 28426 ts_manager.cc:194] Registered new tserver with Master: 8656ec78e9d6430d94d4fa718a116b3c (127.15.212.67:38087)
I20260928 23:09:04.268703 28426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:49841
I20260928 23:09:04.269953 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:04.270506 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28507
I20260928 23:09:04.276859 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28644
I20260928 23:09:04.282787 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28781
I20260928 23:09:04.287918 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28394
I20260928 23:09:04.293226 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34113
--webserver_interface=127.15.212.126
--webserver_port=46123
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34113
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:04.371762 28913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:04.371943 28913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:04.371964 28913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:04.373055 28913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:04.373095 28913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:04.373107 28913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:04.373119 28913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:04.373131 28913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:04.375109 28913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34113
--ranger_config_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=46123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:04.375284 28913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:04.375424 28913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:04.379931 28918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.380009 28919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.380024 28913 server_base.cc:1034] running on GCE node
W20260928 23:09:04.379951 28921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.380422 28913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:04.380780 28913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:04.381917 28913 hybrid_clock.cc:648] HybridClock initialized: now 1790636944381905 us; error 28 us; skew 500 ppm
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:04.384230 28913 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:04.384652 28913 webserver.cc:460] Webserver started at http://127.15.212.126:46123/ using document root <none> and password file <none>
I20260928 23:09:04.384831 28913 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:04.384877 28913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:04.431677 28913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:09:04.432546 28929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.432767 28913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:04.432880 28913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "3a5441fc21754e74a96e2455296f1071"
format_stamp: "Formatted at 2026-09-28 23:09:02 on dist-test-slave-x2t8"
server_key: "e298557221c0dcc2bbecef006c05f3e0"
server_key_iv: "ef1028f53983a6885eff01909b2d9cc9"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.433243 28913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:04.445488 28913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:04.447410 28913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:04.447574 28913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:04.451666 28913 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:34113
I20260928 23:09:04.451711 28981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:34113 every 8 connection(s)
I20260928 23:09:04.452060 28913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:04.453325 28982 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:09:04.453966 28982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Bootstrap starting.
I20260928 23:09:04.458153 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28913
I20260928 23:09:04.458467 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:33123
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:34113
--webserver_port=38841
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:04.462486 28982 log.cc:824] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:04.464130 28982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:04.464406 28982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Bootstrap complete.
I20260928 23:09:04.466754 28982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:04.467005 28982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5441fc21754e74a96e2455296f1071, State: Initialized, Role: FOLLOWER
I20260928 23:09:04.467221 28982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:04.467324 28982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:04.467386 28982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:04.467435 28982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:09:04.468395 28982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:04.468528 28982 leader_election.cc:302] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a5441fc21754e74a96e2455296f1071; no voters:
I20260928 23:09:04.468717 28982 leader_election.cc:288] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 23:09:04.468780 28986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 23:09:04.468928 28986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [term 2 LEADER]: Becoming Leader. State: Replica: 3a5441fc21754e74a96e2455296f1071, State: Running, Role: LEADER
I20260928 23:09:04.468982 28982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:04.469027 28986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } }
I20260928 23:09:04.469281 28988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a5441fc21754e74a96e2455296f1071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } } }
I20260928 23:09:04.469280 28986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a5441fc21754e74a96e2455296f1071. Latest consensus state: current_term: 2 leader_uuid: "3a5441fc21754e74a96e2455296f1071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a5441fc21754e74a96e2455296f1071" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34113 } } }
I20260928 23:09:04.469481 28986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:04.469465 28988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:04.469944 28995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:04.470407 28995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:04.470518 28995 catalog_manager.cc:1259] Loaded cluster ID: 69d1b97c1e744b3d97a047ae073242b6
I20260928 23:09:04.470549 28995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:04.470799 28982 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 23:09:04.474023 28995 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:04.474581 28995 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Loaded TSK: 0
I20260928 23:09:04.474906 28995 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 23:09:04.547181 28984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:04.547462 28984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:04.547500 28984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:04.548931 28984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:04.548998 28984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:04.549042 28984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:04.550576 28984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:33123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=38841
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:04.550830 28984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:04.551062 28984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:04.556155 29022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.556241 29020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.556154 29019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.556810 28984 server_base.cc:1034] running on GCE node
I20260928 23:09:04.557016 28984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:04.557274 28984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:04.560508 28984 hybrid_clock.cc:648] HybridClock initialized: now 1790636944560474 us; error 44 us; skew 500 ppm
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:04.562994 28984 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:04.563499 28984 webserver.cc:460] Webserver started at http://127.15.212.65:38841/ using document root <none> and password file <none>
I20260928 23:09:04.563731 28984 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:04.563812 28984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:04.592840 28984 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260928 23:09:04.593979 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.594144 28984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:04.594236 28984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "cb55d9e741514a79abe8dc081c3b8a5f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "aa0719bb359550a2afaa84b0cf72652d"
server_key_iv: "b9fd9a3b01b700a3abb50f33d4c839c6"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.594712 28984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:04.609746 28984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:04.612128 28984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:04.612298 28984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:04.612552 28984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:04.612914 28984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:04.612969 28984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.613006 28984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:04.613029 28984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.619588 28984 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:33123
I20260928 23:09:04.620061 28984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:04.623489 29143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:33123 every 8 connection(s)
I20260928 23:09:04.624085 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 28984
I20260928 23:09:04.624396 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:42271
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:34113
--webserver_port=37825
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:04.641969 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.624161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.65:49419)
0928 23:09:04.627122 (+ 2961us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:04.627169 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:04.627802 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:04.628641 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:04.628645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:04.628870 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:04.629155 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.629162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.630859 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.630860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.632161 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.632164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.633187 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.633189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.633320 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.633934 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:04.633944 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:04.635982 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:04.638449 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.638455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.638457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.638732 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.640699 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.640700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.640701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.640831 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.641297 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:04.641303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:04.641463 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 23:09:04.641573 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 23:09:04.641726 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2910,"thread_start_us":81,"threads_started":1}
I20260928 23:09:04.642517 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.624817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:49419 (local address 127.15.212.126:34113)
0928 23:09:04.628020 (+ 3203us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:04.628029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:04.628066 (+ 37us) server_negotiation.cc:407] Connection header received
0928 23:09:04.628222 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:04.628228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:04.628284 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:04.628411 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:04.629291 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.630740 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.632261 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.632952 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.633072 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.636121 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:04.636126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:04.636126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:04.636138 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:04.638341 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.640511 (+ 2170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.640514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.640514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.640616 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.640902 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.640902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.640903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.641189 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:04.641259 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:04.642171 (+ 912us) server_negotiation.cc:299] Negotiation successful
0928 23:09:04.642294 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3147,"thread_start_us":80,"threads_started":1}
I20260928 23:09:04.643169 29144 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:04.643272 29144 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.643473 29144 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:04.644459 29144 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:34113 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 23:09:04.644536 29144 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.644608 29144 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:04.645074 29144 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.645152 29144 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:04.645517 29144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:09:04.757876 29148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:04.758153 29148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:04.758277 29148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:04.760001 29148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:04.760118 29148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:04.760207 29148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:04.762331 29148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:42271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=37825
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:04.762692 29148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:04.762946 29148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:04.769268 29153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.769486 29156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.769268 29154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.771546 29148 server_base.cc:1034] running on GCE node
I20260928 23:09:04.771780 29148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:04.772053 29148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:04.773205 29148 hybrid_clock.cc:648] HybridClock initialized: now 1790636944773186 us; error 33 us; skew 500 ppm
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:04.776352 29148 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:04.776907 29148 webserver.cc:460] Webserver started at http://127.15.212.66:37825/ using document root <none> and password file <none>
I20260928 23:09:04.777171 29148 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:04.777232 29148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:04.798084 29148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:04.799469 29164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.799640 29148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:04.799737 29148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f"
format_stamp: "Formatted at 2026-09-28 23:09:03 on dist-test-slave-x2t8"
server_key: "67e1ea4c028bf61a524bbcaca461efc3"
server_key_iv: "4ddc597133c15d55cd140ee7e77305d0"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:04.800251 29148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:04.813350 29148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:04.815822 29148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:04.816041 29148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:04.816329 29148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:04.816743 29148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:04.816830 29148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.816901 29148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:04.816972 29148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:04.823201 29148 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:42271
I20260928 23:09:04.823705 29148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:04.826500 29277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:42271 every 8 connection(s)
I20260928 23:09:04.832181 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29148
I20260928 23:09:04.832484 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.212.84:47809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:38087
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:34113
--webserver_port=44419
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:04.840175 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.827099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:48453 (local address 127.15.212.126:34113)
0928 23:09:04.827148 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:04.827150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:04.827736 (+ 586us) server_negotiation.cc:407] Connection header received
0928 23:09:04.828404 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:04.828405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:04.828417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:04.828482 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:04.829255 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.830059 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.831580 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.831935 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.831979 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.836783 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:04.836785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:04.836785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:04.836794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:04.838493 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.839025 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.839027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.839028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.839091 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:04.839392 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:04.839393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:04.839394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:04.839533 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:04.839581 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:04.839990 (+ 409us) server_negotiation.cc:299] Negotiation successful
0928 23:09:04.840076 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:09:04.840934 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:04.827271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.66:48453)
0928 23:09:04.827613 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:04.827654 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:04.828277 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:04.828664 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:04.828668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:04.828877 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:04.829145 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.829150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.830172 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.830174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.831447 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:04.831450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:04.834246 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:04.834247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:04.834397 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:04.834947 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:04.834957 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:04.836666 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:04.838574 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.838580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.838582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.838907 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.839171 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:04.839172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:04.839172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:04.839305 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:04.839635 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:04.839641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:04.839805 (+ 164us) client_negotiation.cc:769] Sending connection context
0928 23:09:04.840536 (+ 731us) client_negotiation.cc:240] Negotiation successful
0928 23:09:04.840698 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":77,"threads_started":1}
I20260928 23:09:04.841879 29278 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:04.841997 29278 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.842192 29278 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:04.842872 29278 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:34113 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 23:09:04.842958 29278 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.843032 29278 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:04.843349 29278 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:04.843430 29278 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:04.843746 29278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:09:04.961318 29281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:04.961556 29281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:04.961597 29281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:04.962756 29281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:04.962829 29281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:04.962872 29281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:04.965029 29281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.212.84:47809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:38087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=44419
--tserver_master_addrs=127.15.212.126:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:04.965472 29281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:04.965771 29281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:04.972358 29287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.972746 29286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:04.976917 29289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:04.977784 29281 server_base.cc:1034] running on GCE node
I20260928 23:09:04.977991 29281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:04.978248 29281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:04.979408 29281 hybrid_clock.cc:648] HybridClock initialized: now 1790636944979375 us; error 47 us; skew 500 ppm
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:04.982697 29281 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:04.983202 29281 webserver.cc:460] Webserver started at http://127.15.212.67:44419/ using document root <none> and password file <none>
I20260928 23:09:04.983465 29281 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:04.983533 29281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 23:09:04 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:05.004993 29281 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260928 23:09:05.006096 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:05.006296 29281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:05.006378 29281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "8656ec78e9d6430d94d4fa718a116b3c"
format_stamp: "Formatted at 2026-09-28 23:09:04 on dist-test-slave-x2t8"
server_key: "f3b458f87ab9fec8d4b315a7b18eac03"
server_key_iv: "b46393138bf64fee05e19fbded2c6174"
server_key_version: "kuduclusterkey@0"
I20260928 23:09:05.006840 29281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:05.035032 29281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:05.037701 29281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:05.037915 29281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:05.038182 29281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:05.038602 29281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:05.038692 29281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:05.038774 29281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:05.038831 29281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:05.046284 29281 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:38087
I20260928 23:09:05.046787 29281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:09:05.048749 29411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:38087 every 8 connection(s)
I20260928 23:09:05.056134 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29281
Sep 28 23:09:05 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636944, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:05.064520 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:05.049373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34113 (local address 127.15.212.67:38103)
0928 23:09:05.050278 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:05.050339 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:05.053361 (+ 3022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:05.053717 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:05.053722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:05.053939 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:05.054254 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:05.054260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.055202 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.055203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:05.058080 (+ 2877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:05.058082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.058683 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.058685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:05.058825 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:05.059379 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:05.059398 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:05.061041 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:05.062967 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:05.062973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:05.062975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:05.063247 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:05.063466 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:05.063466 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:05.063467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:05.063589 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:05.063868 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:05.063873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:05.064023 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 23:09:05.064118 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 23:09:05.064257 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":87,"threads_started":1}
I20260928 23:09:05.065151 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:05.049644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:38103 (local address 127.15.212.126:34113)
0928 23:09:05.049687 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:05.049689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:05.050426 (+ 737us) server_negotiation.cc:407] Connection header received
0928 23:09:05.053475 (+ 3049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:05.053481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:05.053493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:05.053584 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:05.054361 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.055085 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.058180 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.058520 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.058598 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:05.061148 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:05.061149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:05.061150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:05.061157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:05.062873 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:05.063329 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:05.063330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:05.063331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:05.063386 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:05.063657 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:05.063659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:05.063659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:05.063786 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:05.063844 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:05.064975 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0928 23:09:05.065058 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:09:05.065688 29412 heartbeater.cc:347] Connected to a master server at 127.15.212.126:34113
I20260928 23:09:05.065788 29412 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:05.065966 29412 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:05.066601 29412 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:34113 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 23:09:05.066670 29412 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:05.066727 29412 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:05.067050 29412 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:05.067124 29412 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
W20260928 23:09:05.067417 29412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 23:09:05.074501 29416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:05.057379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.15.212.126:34113)
0928 23:09:05.058399 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:05.058401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:05.058408 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:09:05.058429 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:05.058429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:05.058451 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:05.059215 (+ 764us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:05.059246 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.062269 (+ 3023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.063370 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:05.063680 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:05.065091 (+ 1411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:05.065137 (+ 46us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:05.065138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:05.065138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:05.065142 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:05.068071 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:05.072531 (+ 4460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:05.072532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:05.072533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:05.072594 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:05.072827 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:05.072828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:05.072828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:05.072963 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:05.074279 (+ 1316us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:05.074322 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 23:09:05.074401 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":86,"threads_started":1}
Sep 28 23:09:05 dist-test-slave-x2t8 krb5kdc[27950](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790636945, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:05.645695 29144 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:05.645846 29144 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:05.646220 28946 ts_manager.cc:194] Registered new tserver with Master: cb55d9e741514a79abe8dc081c3b8a5f (127.15.212.65:33123)
I20260928 23:09:05.647184 28946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:49419
I20260928 23:09:05.843936 29278 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:05.844096 29278 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:05.844611 28946 ts_manager.cc:194] Registered new tserver with Master: ac64b8c72ff541b8b13e88bfd6a6b37f (127.15.212.66:42271)
I20260928 23:09:05.845184 28946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:48453
Sep 28 23:09:05 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636945, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:06.067597 29412 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:06.067747 29412 heartbeater.cc:511] Master 127.15.212.126:34113 requested a full tablet report, sending...
I20260928 23:09:06.068058 28946 ts_manager.cc:194] Registered new tserver with Master: 8656ec78e9d6430d94d4fa718a116b3c (127.15.212.67:38087)
I20260928 23:09:06.068495 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:06.068624 28946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:38103
I20260928 23:09:06.081661 29433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.069436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.15.212.126:34113)
0928 23:09:06.069762 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.069764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.069862 (+ 98us) server_negotiation.cc:407] Connection header received
0928 23:09:06.069914 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.069915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.069928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.070050 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:06.070729 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.071531 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.072589 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.072929 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.072993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.078516 (+ 5523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:06.078517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:06.078518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:06.078525 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:06.080379 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:06.080798 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:06.080799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:06.080800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:06.080864 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:06.081104 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:06.081105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:06.081105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:06.081236 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:06.081335 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.081478 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.081559 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":62,"threads_started":1}
I20260928 23:09:06.083937 28946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:09:06.086131 28946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:09:06.096765 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:42271 (local address 127.0.0.1:40870)
0928 23:09:06.089480 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.089518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:06.089584 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:06.092837 (+ 3253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:06.092840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:06.092842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:06.092903 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.092905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.094166 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.094167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.095444 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.095447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.096482 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.096483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.096560 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.096574 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 23:09:06.096631 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 23:09:06.096682 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":91,"threads_started":1}
I20260928 23:09:06.097376 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.15.212.66:42271)
0928 23:09:06.092578 (+ 3118us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.092583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.092594 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:09:06.092630 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.092633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.092648 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.092734 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:06.093001 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.094067 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.095544 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.096366 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.097161 (+ 795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.097229 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.097256 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.097298 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3081,"thread_start_us":67,"threads_started":1}
I20260928 23:09:06.098287 29212 tablet_service.cc:1467] Processing CreateTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 (DEFAULT_TABLE table=test-table [id=1abef30839fd4aa99bd96a5a62b83323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:06.098662 29212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66fee47fdcab4023a92ef64b735a7fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:06.100463 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.15.212.67:38087)
0928 23:09:06.089836 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.089874 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.092671 (+ 2797us) server_negotiation.cc:407] Connection header received
0928 23:09:06.092709 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.092712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.092728 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.092794 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:06.093107 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.094436 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.096106 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.097195 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.097263 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.097314 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.100311 (+ 2997us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.100353 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":60,"threads_started":1}
I20260928 23:09:06.101742 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.15.212.65:33123)
0928 23:09:06.089766 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.089771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.092952 (+ 3181us) server_negotiation.cc:407] Connection header received
0928 23:09:06.094616 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.094619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.094635 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.094686 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:06.097373 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.098444 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.100577 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.101316 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.101365 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.101412 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.101624 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.101664 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":47,"threads_started":1}
I20260928 23:09:06.099980 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:38087 (local address 127.0.0.1:36816)
0928 23:09:06.092535 (+ 3436us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.092546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:06.092606 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:06.092982 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:06.092983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:06.092985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:06.093023 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.093025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.094885 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.094885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.095998 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.096001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.099771 (+ 3770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.099772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.099826 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.099830 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:06.099874 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 23:09:06.099910 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3419,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260928 23:09:06.101985 29441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.089280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:33123 (local address 127.0.0.1:44250)
0928 23:09:06.092892 (+ 3612us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.092901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:06.094519 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:06.094789 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:06.094789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:06.094791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:06.094827 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.094828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.098547 (+ 3719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.098547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.099686 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.099689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.101475 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.101476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.101535 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.101539 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:06.101812 (+ 273us) client_negotiation.cc:240] Negotiation successful
0928 23:09:06.101850 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3598,"thread_start_us":56,"threads_started":1}
I20260928 23:09:06.102615 29346 tablet_service.cc:1467] Processing CreateTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 (DEFAULT_TABLE table=test-table [id=1abef30839fd4aa99bd96a5a62b83323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:06.102614 29078 tablet_service.cc:1467] Processing CreateTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 (DEFAULT_TABLE table=test-table [id=1abef30839fd4aa99bd96a5a62b83323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:06.102936 29346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66fee47fdcab4023a92ef64b735a7fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:06.102957 29078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66fee47fdcab4023a92ef64b735a7fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:06.106207 29445 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Bootstrap starting.
I20260928 23:09:06.106844 29445 tablet_bootstrap.cc:654] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:06.107167 29445 log.cc:824] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:06.107525 29447 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Bootstrap starting.
I20260928 23:09:06.107957 29445 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: No bootstrap required, opened a new log
I20260928 23:09:06.108047 29445 ts_tablet_manager.cc:1397] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:09:06.109022 29447 tablet_bootstrap.cc:654] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:06.109297 29447 log.cc:824] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:06.109583 29446 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Bootstrap starting.
I20260928 23:09:06.110167 29446 tablet_bootstrap.cc:654] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:06.110455 29446 log.cc:824] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:06.111141 29445 raft_consensus.cc:348] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.111313 29445 raft_consensus.cc:374] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:06.111382 29445 raft_consensus.cc:729] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac64b8c72ff541b8b13e88bfd6a6b37f, State: Initialized, Role: FOLLOWER
I20260928 23:09:06.111512 29445 consensus_queue.cc:260] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.111879 29445 ts_tablet_manager.cc:1428] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 23:09:06.112005 29278 heartbeater.cc:503] Master 127.15.212.126:34113 was elected leader, sending a full tablet report...
I20260928 23:09:06.112831 29447 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: No bootstrap required, opened a new log
I20260928 23:09:06.112902 29447 ts_tablet_manager.cc:1397] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 23:09:06.115356 29447 raft_consensus.cc:348] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.115526 29447 raft_consensus.cc:374] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:06.115566 29447 raft_consensus.cc:729] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8656ec78e9d6430d94d4fa718a116b3c, State: Initialized, Role: FOLLOWER
I20260928 23:09:06.115721 29447 consensus_queue.cc:260] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.115974 29447 ts_tablet_manager.cc:1428] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:09:06.116006 29412 heartbeater.cc:503] Master 127.15.212.126:34113 was elected leader, sending a full tablet report...
I20260928 23:09:06.116678 29446 tablet_bootstrap.cc:492] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: No bootstrap required, opened a new log
I20260928 23:09:06.116792 29446 ts_tablet_manager.cc:1397] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260928 23:09:06.119269 29446 raft_consensus.cc:348] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.119414 29446 raft_consensus.cc:374] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:06.119444 29446 raft_consensus.cc:729] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb55d9e741514a79abe8dc081c3b8a5f, State: Initialized, Role: FOLLOWER
I20260928 23:09:06.119563 29446 consensus_queue.cc:260] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.119796 29446 ts_tablet_manager.cc:1428] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:09:06.119838 29144 heartbeater.cc:503] Master 127.15.212.126:34113 was elected leader, sending a full tablet report...
W20260928 23:09:06.126902 29145 tablet.cc:2369] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:06.157902 29453 raft_consensus.cc:482] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:06.158154 29453 raft_consensus.cc:504] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.159099 29453 leader_election.cc:288] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac64b8c72ff541b8b13e88bfd6a6b37f (127.15.212.66:42271), 8656ec78e9d6430d94d4fa718a116b3c (127.15.212.67:38087)
I20260928 23:09:06.163339 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.159035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:56529 (local address 127.15.212.67:38087)
0928 23:09:06.159079 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.159081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.159352 (+ 271us) server_negotiation.cc:407] Connection header received
0928 23:09:06.159785 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.159786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.159796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.159825 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:06.160185 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.160926 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.162362 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.162928 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.162979 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.162988 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.163230 (+ 242us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.163276 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:09:06.163769 29455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.158801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:38087 (local address 127.15.212.65:56529)
0928 23:09:06.159285 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.159297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:06.159527 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:06.160055 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:06.160056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:06.160057 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:06.160109 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.160110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.161030 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.161030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.162256 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.162259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.163072 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.163073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.163144 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.163148 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:06.163622 (+ 474us) client_negotiation.cc:240] Negotiation successful
0928 23:09:06.163662 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":118,"threads_started":1}
I20260928 23:09:06.164211 29365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66fee47fdcab4023a92ef64b735a7fe3" candidate_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8656ec78e9d6430d94d4fa718a116b3c" is_pre_election: true
I20260928 23:09:06.164366 29365 raft_consensus.cc:2454] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb55d9e741514a79abe8dc081c3b8a5f in term 0.
I20260928 23:09:06.165032 29031 leader_election.cc:302] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8656ec78e9d6430d94d4fa718a116b3c, cb55d9e741514a79abe8dc081c3b8a5f; no voters:
I20260928 23:09:06.165223 29453 raft_consensus.cc:2781] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:06.165277 29453 raft_consensus.cc:482] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:06.165300 29453 raft_consensus.cc:3037] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:06.166203 29453 raft_consensus.cc:504] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.166636 29365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66fee47fdcab4023a92ef64b735a7fe3" candidate_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8656ec78e9d6430d94d4fa718a116b3c"
I20260928 23:09:06.166712 29365 raft_consensus.cc:3037] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:06.166924 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.159029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:58563 (local address 127.15.212.66:42271)
0928 23:09:06.159084 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.159086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.159495 (+ 409us) server_negotiation.cc:407] Connection header received
0928 23:09:06.159667 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.159669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.159678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.159709 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:06.160714 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.161444 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.163586 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.164173 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.164224 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.164232 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.166804 (+ 2572us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.166853 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:09:06.167171 29454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.158805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:42271 (local address 127.15.212.65:58563)
0928 23:09:06.159430 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.159439 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:06.159585 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:06.159892 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:06.159892 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:06.159894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:06.159973 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.159975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.162352 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.162352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.163482 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:06.163485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.166632 (+ 3147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.166633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:06.166713 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.166717 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:06.167037 (+ 320us) client_negotiation.cc:240] Negotiation successful
0928 23:09:06.167089 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":120,"threads_started":1}
I20260928 23:09:06.167464 29232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66fee47fdcab4023a92ef64b735a7fe3" candidate_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f"
I20260928 23:09:06.167570 29365 raft_consensus.cc:2454] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb55d9e741514a79abe8dc081c3b8a5f in term 1.
I20260928 23:09:06.167573 29232 raft_consensus.cc:3037] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:06.168012 29031 leader_election.cc:302] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8656ec78e9d6430d94d4fa718a116b3c, cb55d9e741514a79abe8dc081c3b8a5f; no voters:
I20260928 23:09:06.168239 29453 leader_election.cc:288] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [CANDIDATE]: Term 1 election: Requested vote from peers ac64b8c72ff541b8b13e88bfd6a6b37f (127.15.212.66:42271), 8656ec78e9d6430d94d4fa718a116b3c (127.15.212.67:38087)
I20260928 23:09:06.168325 29453 raft_consensus.cc:2781] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:06.168485 29232 raft_consensus.cc:2454] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb55d9e741514a79abe8dc081c3b8a5f in term 1.
I20260928 23:09:06.168556 29453 raft_consensus.cc:686] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [term 1 LEADER]: Becoming Leader. State: Replica: cb55d9e741514a79abe8dc081c3b8a5f, State: Running, Role: LEADER
I20260928 23:09:06.168648 29453 consensus_queue.cc:237] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } }
I20260928 23:09:06.167469 29230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66fee47fdcab4023a92ef64b735a7fe3" candidate_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" is_pre_election: true
I20260928 23:09:06.168823 29230 raft_consensus.cc:2362] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb55d9e741514a79abe8dc081c3b8a5f in term 1. Re-sending same reply.
I20260928 23:09:06.169458 28945 catalog_manager.cc:5495] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb55d9e741514a79abe8dc081c3b8a5f (127.15.212.65). New cstate: current_term: 1 leader_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb55d9e741514a79abe8dc081c3b8a5f" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 } health_report { overall_health: UNKNOWN } } }
I20260928 23:09:06.188975 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.185548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.15.212.65:33123)
0928 23:09:06.185598 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:06.185600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:06.185610 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:09:06.185698 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:06.185699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:06.185720 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:06.185803 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:06.185956 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.186689 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.187836 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:06.188103 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:06.188182 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:06.188361 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:06.188545 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:06.188640 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:06.188878 (+ 238us) server_negotiation.cc:299] Negotiation successful
0928 23:09:06.188900 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 23:09:06.194458 29365 raft_consensus.cc:1261] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 1 FOLLOWER]: Refusing update from remote peer cb55d9e741514a79abe8dc081c3b8a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:06.194832 29453 consensus_queue.cc:1035] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8656ec78e9d6430d94d4fa718a116b3c" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:06.195300 29230 raft_consensus.cc:1261] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 1 FOLLOWER]: Refusing update from remote peer cb55d9e741514a79abe8dc081c3b8a5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:06.196611 29453 consensus_queue.cc:1035] T 66fee47fdcab4023a92ef64b735a7fe3 P cb55d9e741514a79abe8dc081c3b8a5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac64b8c72ff541b8b13e88bfd6a6b37f" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 42271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:06.199102 29460 mvcc.cc:205] Tried to move back new op lower bound from 7334448931596255232 to 7334448931507503104. Current Snapshot: MvccSnapshot[applied={T|T < 7334448931596255232}]
I20260928 23:09:06.199350 29461 mvcc.cc:205] Tried to move back new op lower bound from 7334448931596255232 to 7334448931507503104. Current Snapshot: MvccSnapshot[applied={T|T < 7334448931596255232}]
Sep 28 23:09:06 dist-test-slave-x2t8 krb5kdc[27950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636945, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for HTTP/127.15.212.84@KRBTEST.COM
I20260928 23:09:06.202219 28945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:06.202301 28945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:06.203061 28945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a5441fc21754e74a96e2455296f1071: Sending DeleteTablet for 3 replicas of tablet 66fee47fdcab4023a92ef64b735a7fe3
I20260928 23:09:06.203380 29346 tablet_service.cc:1514] Processing DeleteTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:06 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36816
I20260928 23:09:06.203459 29212 tablet_service.cc:1514] Processing DeleteTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:06 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:40870
I20260928 23:09:06.203707 29468 tablet_replica.cc:330] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: stopping tablet replica
I20260928 23:09:06.203828 29468 raft_consensus.cc:2229] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:06.204030 29078 tablet_service.cc:1514] Processing DeleteTablet for tablet 66fee47fdcab4023a92ef64b735a7fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:06 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:44250
I20260928 23:09:06.204581 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28984
I20260928 23:09:06.204725 29463 mvcc.cc:205] Tried to move back new op lower bound from 7334448931596255232 to 7334448931507503104. Current Snapshot: MvccSnapshot[applied={T|T < 7334448931596255232}]
I20260928 23:09:06.207361 29468 raft_consensus.cc:2258] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:06.207846 29468 ts_tablet_manager.cc:1905] T 66fee47fdcab4023a92ef64b735a7fe3 P 8656ec78e9d6430d94d4fa718a116b3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:06.208935 29469 tablet_replica.cc:330] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: stopping tablet replica
I20260928 23:09:06.209088 29469 raft_consensus.cc:2229] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:06.209246 29469 raft_consensus.cc:2258] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:06.212790 29469 ts_tablet_manager.cc:1905] T 66fee47fdcab4023a92ef64b735a7fe3 P ac64b8c72ff541b8b13e88bfd6a6b37f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 23:09:06.213372 28931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:33123 (error 108)
I20260928 23:09:06.213920 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29148
I20260928 23:09:06.219447 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29281
I20260928 23:09:06.225674 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 28913
I20260928 23:09:06.225780 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:06.225585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:33123 (local address 127.0.0.1:44282)
0928 23:09:06.225704 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:06.225742 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:33123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260928 23:09:06.232343 16209 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 23:09:06.576587 16209 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 23:09:06.576689 16209 mini_ranger.cc:66] Stopping Ranger...
I20260928 23:09:06.921190 16209 mini_ranger.cc:68] Stopped Ranger
2026-09-28 23:09:06.921 UTC [27980] LOG: received smart shutdown request
2026-09-28 23:09:06.925 UTC [27980] LOG: background worker "logical replication launcher" (PID 27989) exited with exit code 1
2026-09-28 23:09:06.925 UTC [27984] LOG: shutting down
2026-09-28 23:09:07.089 UTC [27980] LOG: database system is shut down
2026-09-28T23:09:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46661 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:09:07 dist-test-slave-x2t8 krb5kdc[29482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:09:07 dist-test-slave-x2t8 krb5kdc[29482](info): set up 2 sockets
Sep 28 23:09:07 dist-test-slave-x2t8 krb5kdc[29482](info): commencing operation
krb5kdc: starting...
W20260928 23:09:08.244349 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:09:08.356145 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45005
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:08.434152 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:08.434347 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:08.434369 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:08.435489 29495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:08.435530 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:08.435545 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:08.435557 29495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:08.435621 29495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:08.437067 29495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:08.437320 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:08.437585 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:08.442505 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:08.442534 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.442534 29495 server_base.cc:1034] running on GCE node
W20260928 23:09:08.442516 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.443048 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:08.443377 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:08.444499 29495 hybrid_clock.cc:648] HybridClock initialized: now 1790636948444482 us; error 46 us; skew 500 ppm
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:08.446987 29495 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:08.447397 29495 webserver.cc:460] Webserver started at http://127.15.212.126:37175/ using document root <none> and password file <none>
I20260928 23:09:08.447579 29495 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:08.447628 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:08.447717 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:08.448813 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "7003acc76bb149e68ce2f3b47efbce3b"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "1802f2c4e8b3d6ab271363f312716f5d"
server_key_iv: "b12876ba2e7fac6c39dc7410bf92d211"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.449126 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "7003acc76bb149e68ce2f3b47efbce3b"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "1802f2c4e8b3d6ab271363f312716f5d"
server_key_iv: "b12876ba2e7fac6c39dc7410bf92d211"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.450378 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:09:08.451066 29510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:08.451256 29495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:08.451344 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "7003acc76bb149e68ce2f3b47efbce3b"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "1802f2c4e8b3d6ab271363f312716f5d"
server_key_iv: "b12876ba2e7fac6c39dc7410bf92d211"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.451399 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:08.459189 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:08.461171 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:08.461323 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:08.465050 29495 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:45005
I20260928 23:09:08.465082 29562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:45005 every 8 connection(s)
I20260928 23:09:08.465426 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:08.466284 29563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:08.467760 29563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.469885 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29495
I20260928 23:09:08.470021 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:09:08.470201 16209 external_mini_cluster.cc:1428] Setting key 3228d8eec299fc810d3949d9385b4577
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.212.125@KRBTEST.COM; defaulting to no policy
I20260928 23:09:08.478103 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.468406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.15.212.126:45005)
0928 23:09:08.468624 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.468634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.468733 (+ 99us) server_negotiation.cc:407] Connection header received
0928 23:09:08.469279 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.469286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.469303 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.469406 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.470090 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.470902 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.471985 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.472294 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.472420 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.474904 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.474911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.474911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.474917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.476492 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.476933 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.476935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.476936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.477008 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.477288 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.477288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.477289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.477574 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.477646 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.477818 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.477967 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":179,"thread_start_us":133,"threads_started":1}
I20260928 23:09:08.478122 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.468379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:45005 (local address 127.0.0.1:50322)
0928 23:09:08.468614 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.468656 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.469124 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.469413 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.469415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.469737 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.469970 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.469977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.471003 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.471004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.471879 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.471882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.472392 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.472394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.472482 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.473016 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.473030 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.474817 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.476574 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.476581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.476584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.476843 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.477072 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.477073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.477074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.477198 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.477641 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.477646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.477701 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.477777 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.477897 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":119,"threads_started":1}
I20260928 23:09:08.478931 29563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.479279 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.479170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37484)
0928 23:09:08.479221 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.479261 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:09:08.479439 29512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.125@KRBTEST.COM" created.
I20260928 23:09:08.479873 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.479770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37486)
0928 23:09:08.479820 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.479847 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:09:08.479991 29563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:36299: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
I20260928 23:09:08.495436 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:36299
--webserver_interface=127.15.212.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:08.529244 29563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } attempt: 1
I20260928 23:09:08.529768 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.529583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37498)
0928 23:09:08.529657 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.529715 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 23:09:08.530215 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.530075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37514)
0928 23:09:08.530142 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.530181 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 23:09:08.530344 29563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:36299: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
W20260928 23:09:08.574190 29570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:08.574383 29570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:08.574404 29570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:08.575414 29570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:08.575445 29570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:08.575459 29570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:08.575470 29570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:08.575482 29570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:08.576886 29570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:36299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.212.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:08.577064 29570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:08.577219 29570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:08.581908 29575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.582029 29570 server_base.cc:1034] running on GCE node
W20260928 23:09:08.581923 29576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:08.582126 29578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.582418 29570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:08.582744 29570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:08.583910 29570 hybrid_clock.cc:648] HybridClock initialized: now 1790636948583899 us; error 43 us; skew 500 ppm
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:08.586270 29570 init.cc:377] Logged in from keytab as kudu/127.15.212.125@KRBTEST.COM (short username kudu)
I20260928 23:09:08.586678 29570 webserver.cc:460] Webserver started at http://127.15.212.125:37725/ using document root <none> and password file <none>
I20260928 23:09:08.586892 29570 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:08.586958 29570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:08.587105 29570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:08.588093 29570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data/instance:
uuid: "c07190802cc24ec1b83f55c8f84fbf98"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "0c668e0a0f39b2cbba8e4dd9c5205962"
server_key_iv: "70c64139e362f541b7d9d97c4668f671"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.588498 29570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal/instance:
uuid: "c07190802cc24ec1b83f55c8f84fbf98"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "0c668e0a0f39b2cbba8e4dd9c5205962"
server_key_iv: "70c64139e362f541b7d9d97c4668f671"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.589768 29570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:08.590448 29585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:08.590632 29570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:09:08.590727 29570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "c07190802cc24ec1b83f55c8f84fbf98"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "0c668e0a0f39b2cbba8e4dd9c5205962"
server_key_iv: "70c64139e362f541b7d9d97c4668f671"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.590811 29570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:08.598508 29563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } attempt: 2
I20260928 23:09:08.598901 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.598766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37516)
0928 23:09:08.598823 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.598863 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:09:08.599237 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.599148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37520)
0928 23:09:08.599193 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.599217 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 23:09:08.599349 29563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:36299: Network error: Client connection negotiation failed: client connection to 127.15.212.125:36299: connect: Connection refused (error 111)
I20260928 23:09:08.601004 29570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:08.602695 29570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:08.602849 29570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:08.606705 29570 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.125:36299
I20260928 23:09:08.606737 29637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.125:36299 every 8 connection(s)
I20260928 23:09:08.607065 29570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
I20260928 23:09:08.607861 29638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:08.609352 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.609675 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29570
I20260928 23:09:08.609815 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-1/wal/instance
I20260928 23:09:08.610023 16209 external_mini_cluster.cc:1428] Setting key 264ca420251398e190a467f3ef0a7348
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.212.124@KRBTEST.COM; defaulting to no policy
I20260928 23:09:08.618770 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.610082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.15.212.126:45005)
0928 23:09:08.610135 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.610137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.610333 (+ 196us) server_negotiation.cc:407] Connection header received
0928 23:09:08.610818 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.610820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.610832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.610877 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.611707 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.612387 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.613393 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.613626 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.613715 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.615943 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.615944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.615945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.615951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.617534 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.617929 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.617930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.617930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.617998 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.618219 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.618219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.618220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.618322 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.618371 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.618589 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.618692 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:09:08.618947 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.610064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:45005 (local address 127.0.0.1:50330)
0928 23:09:08.610217 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.610250 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.610707 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.610953 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.610959 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.611287 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.611597 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.611602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.612521 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.612523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.613289 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.613291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.613723 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.613724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.613855 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.614360 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.614368 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.615833 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.617610 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.617616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.617618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.617861 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.618061 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.618067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.618068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.618159 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.618375 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.618379 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.618484 (+ 105us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.618562 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.618698 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.124@KRBTEST.COM" created.
I20260928 23:09:08.619581 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:09:08.627061 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.619836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37524)
0928 23:09:08.619891 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.619903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.619947 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.620243 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.620244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.620349 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.620391 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.620392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.621488 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.621489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.621972 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.621974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.622455 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.622455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.622519 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.622942 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.622946 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.624307 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.626012 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.626013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.626013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.626180 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.626405 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.626405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.626406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.626458 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.626742 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.626743 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.626780 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.626852 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.626948 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 23:09:08.627135 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.619920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37524 (local address 127.15.212.125:36299)
0928 23:09:08.620086 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.620092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.620115 (+ 23us) server_negotiation.cc:407] Connection header received
0928 23:09:08.620138 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.620140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.620155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.620234 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.620489 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.621379 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.622069 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.622367 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.622455 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.624405 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.624412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.624412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.624418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.625911 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.626271 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.626274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.626274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.626329 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.626527 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.626529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.626529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.626671 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.626764 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.626881 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.627013 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260928 23:09:08.627739 29638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.628058 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.627939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52848)
0928 23:09:08.627983 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.628037 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:09:08.628218 29586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
I20260928 23:09:08.628695 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.628596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52860)
0928 23:09:08.628647 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.628672 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 23:09:08.628832 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:40615: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
I20260928 23:09:08.634577 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:40615
--webserver_interface=127.15.212.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:08.678051 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } attempt: 1
I20260928 23:09:08.678498 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.678371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52874)
0928 23:09:08.678431 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.678470 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:09:08.678886 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.678755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52876)
0928 23:09:08.678816 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.678856 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 23:09:08.679001 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:40615: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
I20260928 23:09:08.690505 29563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } attempt: 3
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:09:08.697297 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.690832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37536)
0928 23:09:08.690886 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.690895 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.690957 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.691122 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.691123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.691282 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.691344 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.691345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.692008 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.692009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.692588 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.692590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.692998 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.692999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.693109 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.693593 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.693604 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.694758 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.696285 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.696286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.696286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.696492 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.696687 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.696688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.696688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.696744 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.696983 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.696984 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.697028 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.697114 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.697207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 23:09:08.697295 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.690885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.15.212.125:36299)
0928 23:09:08.690953 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.690955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.690965 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:09:08.691025 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.691026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.691044 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.691116 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.691433 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.691921 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.692669 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.692923 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.693001 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.694840 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.694842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.694842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.694850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.696186 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.696560 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.696561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.696568 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.696621 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.696812 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.696813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.696813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.696918 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.696991 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.697138 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.697207 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:09:08.697680 29563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.697995 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.697899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52888)
0928 23:09:08.697947 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.697972 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:09:08.698426 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.698341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52904)
0928 23:09:08.698387 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.698410 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:09:08.698525 29563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:40615: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
W20260928 23:09:08.714309 29645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:08.714471 29645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:08.714493 29645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:08.715626 29645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:08.715679 29645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:08.715713 29645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:08.715739 29645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:08.715759 29645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:08.717200 29645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45005,127.15.212.125:36299,127.15.212.124:40615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:40615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.212.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:08.717453 29645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:08.717675 29645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:08.722718 29653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:08.722729 29651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.722781 29645 server_base.cc:1034] running on GCE node
W20260928 23:09:08.722728 29650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:08.723138 29645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:08.723441 29645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:08.724589 29645 hybrid_clock.cc:648] HybridClock initialized: now 1790636948724566 us; error 39 us; skew 500 ppm
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:08.726953 29645 init.cc:377] Logged in from keytab as kudu/127.15.212.124@KRBTEST.COM (short username kudu)
I20260928 23:09:08.727350 29645 webserver.cc:460] Webserver started at http://127.15.212.124:42727/ using document root <none> and password file <none>
I20260928 23:09:08.727573 29645 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:08.727638 29645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:08.727775 29645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:08.728819 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data/instance:
uuid: "9798ef71d14d4088b269ecd998d5783c"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "a77a23fbf753af721997b9f314de48a4"
server_key_iv: "749b29bbf404459b6b5f1f43f752eb5d"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.729152 29645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal/instance:
uuid: "9798ef71d14d4088b269ecd998d5783c"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "a77a23fbf753af721997b9f314de48a4"
server_key_iv: "749b29bbf404459b6b5f1f43f752eb5d"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.729657 29563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } attempt: 1
I20260928 23:09:08.729981 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.729871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52912)
0928 23:09:08.729926 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.729959 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 23:09:08.730274 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.730185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52922)
0928 23:09:08.730229 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.730252 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:09:08.730376 29563 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:40615: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
I20260928 23:09:08.730537 29645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:09:08.731268 29660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:08.731428 29645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:08.731523 29645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "9798ef71d14d4088b269ecd998d5783c"
format_stamp: "Formatted at 2026-09-28 23:09:08 on dist-test-slave-x2t8"
server_key: "a77a23fbf753af721997b9f314de48a4"
server_key_iv: "749b29bbf404459b6b5f1f43f752eb5d"
server_key_version: "encryptionkey@0"
I20260928 23:09:08.731601 29645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:08.747216 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } attempt: 2
I20260928 23:09:08.747547 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.747444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52936)
0928 23:09:08.747497 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.747527 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:09:08.747831 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.747746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52940)
0928 23:09:08.747787 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.747809 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 23:09:08.747903 29638 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:40615: Network error: Client connection negotiation failed: client connection to 127.15.212.124:40615: connect: Connection refused (error 111)
I20260928 23:09:08.752111 29645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:08.753897 29645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:08.754050 29645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:08.757911 29645 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.124:40615
I20260928 23:09:08.757941 29712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.124:40615 every 8 connection(s)
I20260928 23:09:08.758260 29645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
I20260928 23:09:08.758932 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29645
I20260928 23:09:08.759053 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790636798761935-16209-0/minicluster-data/master-2/wal/instance
I20260928 23:09:08.759212 29713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:08.759275 16209 external_mini_cluster.cc:1428] Setting key 8d5009d1dd79855833bd93d93ef4628e
I20260928 23:09:08.761523 29713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:08.769204 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.759723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.15.212.124:40615)
0928 23:09:08.759885 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.759891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.759913 (+ 22us) server_negotiation.cc:407] Connection header received
0928 23:09:08.760293 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.760298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.760338 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.760491 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.761214 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.762150 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.763197 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.763605 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.763744 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.765964 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.765971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.765971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.765976 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.767695 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768078 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.768079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.768080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.768159 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768382 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.768383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.768383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.768635 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.768774 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.768883 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.769003 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260928 23:09:08.769342 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.762118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.15.212.126:45005)
0928 23:09:08.762183 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.762185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.762344 (+ 159us) server_negotiation.cc:407] Connection header received
0928 23:09:08.762404 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.762405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.762412 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.762460 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.762967 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.763399 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.764278 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.764531 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.764615 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.766611 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.766612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.766612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.766626 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.768143 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768535 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.768537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.768538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.768596 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768805 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.768806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.768807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.768916 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.768994 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.769183 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.769255 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:09:08.769357 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.762068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:45005 (local address 127.0.0.1:50346)
0928 23:09:08.762242 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.762282 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.762345 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.762504 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.762506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.762824 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.762896 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.762898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.763486 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.763487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.764193 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.764195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.764627 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.764628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.764702 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.765158 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.765172 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.766520 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.768216 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.768225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.768414 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.768670 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.768671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.768671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.768733 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.768986 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.768993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.769041 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.769153 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.769256 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260928 23:09:08.770102 29713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:08.777159 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.770323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:36299 (local address 127.0.0.1:37548)
0928 23:09:08.770373 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.770384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.770418 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.770533 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.770534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.770703 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.770763 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.770764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.771626 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.771627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.772416 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.772418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.772880 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.772881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.772951 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.773318 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.773325 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.774494 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.776135 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.776136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.776136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.776295 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.776552 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.776553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.776553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.776610 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.776859 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.776860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.776900 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.776962 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.777054 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:09:08.777179 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.770410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.15.212.125:36299)
0928 23:09:08.770445 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.770446 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.770453 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:09:08.770472 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.770473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.770481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.770528 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.770834 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.771548 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.772525 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.772804 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.772868 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.774590 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.774592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.774592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.774598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.776040 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.776410 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.776411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.776412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.776489 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.776677 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.776678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.776678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.776780 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.776867 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.777006 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.777086 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:09:08.777544 29713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:09:08.777869 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.770153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.15.212.126:45005)
0928 23:09:08.770189 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.770190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.770199 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:09:08.770251 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.770251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.770261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.770326 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.770934 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.771600 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.772525 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.772804 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.772869 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.775048 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.775049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.775049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.775055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.776522 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.776886 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.776887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.776887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.776935 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.777153 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.777154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.777155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.777278 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.777484 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.777671 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.777766 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:09:08.784650 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.777755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52962)
0928 23:09:08.777798 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.777808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.777841 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.777991 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.777991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.778145 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.778174 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.778175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.778943 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.778943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.779665 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.779667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.780136 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.780136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.780171 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.780569 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.780574 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.781907 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.783622 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.783623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.783623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.783809 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.784049 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.784050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.784050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.784108 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.784365 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.784365 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.784396 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.784524 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.784597 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 23:09:08.784775 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.777807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.15.212.124:40615)
0928 23:09:08.777837 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.777839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.777846 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:09:08.777904 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.777905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.777914 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.777962 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.778236 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.778865 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.779765 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.780047 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.780124 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.781992 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.781993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.781994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.782001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.783542 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.783925 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.783926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.783926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.783980 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.784172 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.784173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.784173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.784294 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.784361 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.784552 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.784654 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":16}
I20260928 23:09:08.785779 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.778472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.15.212.125:36299)
0928 23:09:08.778509 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.778511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.778519 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:08.778592 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.778593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.778601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.778656 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.779038 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.779694 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.780544 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.780867 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.780943 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.783082 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.783083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.783084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.783091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.784720 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.785100 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.785101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.785102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.785163 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.785356 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.785357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.785357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.785455 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.785573 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.785674 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.785728 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:09:08.786764 29713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: Bootstrap starting.
I20260928 23:09:08.787139 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:09:08.787446 29713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:08.787719 29713 log.cc:824] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:08.788648 29713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: No bootstrap required, opened a new log
I20260928 23:09:08.790588 29713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.790738 29713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:08.790773 29713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9798ef71d14d4088b269ecd998d5783c, State: Initialized, Role: FOLLOWER
I20260928 23:09:08.790931 29713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.791198 29713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:08.792721 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
W20260928 23:09:08.793143 29737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:09:08.793593 29737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:09:08.794078 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:09:08.796597 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.788919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.15.212.125:36299)
0928 23:09:08.788950 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.788952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.788960 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:08.789018 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.789020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.789030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.789059 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.789844 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.790521 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.791483 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.791755 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.791792 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.793610 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.793612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.793612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.793619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.795278 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.795630 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.795631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.795632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.795696 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.795877 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.795878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.795878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.795985 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.796072 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.796424 (+ 352us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.796516 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:09:08.794009 29645 master_runner.cc:186] Error getting master registration for 127.15.212.126:45005: OK, instance_id { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" instance_seqno: 1790636948463908 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:09:08.797049 29645 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:09:08.799503 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.788767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.15.212.126:45005)
0928 23:09:08.788809 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.788811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.789101 (+ 290us) server_negotiation.cc:407] Connection header received
0928 23:09:08.789207 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.789208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.789218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.789272 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.789770 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.790475 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.791447 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.791735 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.791792 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.794364 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.794365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.794366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.794372 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.798058 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.798702 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.798704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.798704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.798766 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.798985 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.798986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.798987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.799110 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.799204 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.799323 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.799393 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:09:08.800705 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.789034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.15.212.124:40615)
0928 23:09:08.789101 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.789103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.792887 (+ 3784us) server_negotiation.cc:407] Connection header received
0928 23:09:08.792998 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.792999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.793009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.793037 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.794405 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.795073 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.796053 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.796318 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.796360 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.797735 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.797737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.797737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.797743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.799198 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.799573 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.799574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.799575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.799627 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.800221 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.800222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.800222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.800354 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.800421 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.800601 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.800654 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 23:09:08.805516 29563 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } attempt: 2
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:09:08.811846 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.805766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52984)
0928 23:09:08.805828 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.805836 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.805891 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.806012 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.806012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.806156 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.806183 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.806184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.806936 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.806937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.807489 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.807491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.807851 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.807852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.807909 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.808278 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.808282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.809454 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.811024 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.811024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.811025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.811182 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.811372 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.811373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.811373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.811419 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.811654 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.811655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.811685 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.811734 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.811790 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 23:09:08.811925 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.805830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.15.212.124:40615)
0928 23:09:08.805881 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.805883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.805891 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:08.805947 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.805948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.805956 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.806008 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.806258 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.806858 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.807578 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.807769 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.807848 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.809535 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.809536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.809536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.809543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.810949 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.811251 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.811252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.811253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.811308 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.811476 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.811476 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.811477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.811595 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.811658 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.811754 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.811820 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:09:08.813342 29563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: Bootstrap starting.
I20260928 23:09:08.814033 29563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:08.814328 29563 log.cc:824] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:08.815140 29563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: No bootstrap required, opened a new log
I20260928 23:09:08.817121 29563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.817255 29563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:08.817289 29563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7003acc76bb149e68ce2f3b47efbce3b, State: Initialized, Role: FOLLOWER
I20260928 23:09:08.817440 29563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.817679 29740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.817731 29563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:08.817783 29740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: This master's current role is: FOLLOWER
W20260928 23:09:08.819746 29751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:09:08.819896 29751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:09:08.820150 29495 master_runner.cc:186] Error getting master registration for 127.15.212.125:36299: OK, instance_id { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" instance_seqno: 1790636948605663 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:09:08.821717 29495 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:09:08.839033 29638 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } attempt: 3
Sep 28 23:09:08 dist-test-slave-x2t8 krb5kdc[29482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636948, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:09:08.845629 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.839274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:40615 (local address 127.0.0.1:52988)
0928 23:09:08.839318 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:08.839328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:08.839367 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:08.839511 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:08.839512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:08.839655 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:08.839689 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.839691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.840305 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.840306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.840859 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:08.840860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.841232 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.841233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:08.841294 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.841697 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:08.841701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:08.842853 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:08.844622 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.844623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.844623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.844794 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.845008 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:08.845009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:08.845009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:08.845075 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:08.845316 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:08.845317 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:08.845359 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 23:09:08.845447 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 23:09:08.845537 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260928 23:09:08.845629 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:08.839307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.15.212.124:40615)
0928 23:09:08.839340 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:08.839342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:08.839352 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:09:08.839425 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:08.839426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:08.839440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:08.839507 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:08.839771 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.840222 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.840951 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:08.841134 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:08.841219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:08.842934 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:08.842936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:08.842936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:08.842943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:08.844556 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.844880 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.844881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.844881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.844932 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:08.845138 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:08.845139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:08.845139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:08.845248 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:08.845323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:08.845470 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 23:09:08.845536 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:09:08.846840 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: Bootstrap starting.
I20260928 23:09:08.847440 29638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:08.847741 29638 log.cc:824] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:08.848493 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: No bootstrap required, opened a new log
I20260928 23:09:08.849854 29638 raft_consensus.cc:348] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.849979 29638 raft_consensus.cc:374] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:08.850054 29638 raft_consensus.cc:729] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c07190802cc24ec1b83f55c8f84fbf98, State: Initialized, Role: FOLLOWER
I20260928 23:09:08.850219 29638 consensus_queue.cc:260] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.850464 29754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.850518 29638 sys_catalog.cc:564] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:08.850607 29754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 23:09:08.852766 29765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:09:08.852874 29765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260928 23:09:08.853415 29591 tablet.cc:2369] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:08.855376 29570 master_runner.cc:263] No leader master found from master c07190802cc24ec1b83f55c8f84fbf98
I20260928 23:09:08.855429 29570 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:09:08.889537 29726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:08.889649 29726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.889866 29726 leader_election.cc:288] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7003acc76bb149e68ce2f3b47efbce3b (127.15.212.126:45005), c07190802cc24ec1b83f55c8f84fbf98 (127.15.212.125:36299)
I20260928 23:09:08.890199 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9798ef71d14d4088b269ecd998d5783c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c07190802cc24ec1b83f55c8f84fbf98" is_pre_election: true
I20260928 23:09:08.890326 29612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9798ef71d14d4088b269ecd998d5783c in term 0.
I20260928 23:09:08.890702 29662 leader_election.cc:302] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9798ef71d14d4088b269ecd998d5783c, c07190802cc24ec1b83f55c8f84fbf98; no voters:
I20260928 23:09:08.890936 29726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:08.891003 29726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:08.891036 29726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:08.891870 29726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.892000 29726 leader_election.cc:288] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [CANDIDATE]: Term 1 election: Requested vote from peers 7003acc76bb149e68ce2f3b47efbce3b (127.15.212.126:45005), c07190802cc24ec1b83f55c8f84fbf98 (127.15.212.125:36299)
I20260928 23:09:08.892320 29612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9798ef71d14d4088b269ecd998d5783c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c07190802cc24ec1b83f55c8f84fbf98"
I20260928 23:09:08.892397 29612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:08.893395 29612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9798ef71d14d4088b269ecd998d5783c in term 1.
I20260928 23:09:08.893603 29662 leader_election.cc:302] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9798ef71d14d4088b269ecd998d5783c, c07190802cc24ec1b83f55c8f84fbf98; no voters:
I20260928 23:09:08.894024 29537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9798ef71d14d4088b269ecd998d5783c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7003acc76bb149e68ce2f3b47efbce3b" is_pre_election: true
I20260928 23:09:08.894160 29537 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9798ef71d14d4088b269ecd998d5783c in term 0.
I20260928 23:09:08.894264 29536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9798ef71d14d4088b269ecd998d5783c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7003acc76bb149e68ce2f3b47efbce3b"
I20260928 23:09:08.894310 29536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:08.895119 29536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9798ef71d14d4088b269ecd998d5783c in term 1.
I20260928 23:09:08.896278 29726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:08.896456 29726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [term 1 LEADER]: Becoming Leader. State: Replica: 9798ef71d14d4088b269ecd998d5783c, State: Running, Role: LEADER
I20260928 23:09:08.896540 29726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } }
I20260928 23:09:08.896884 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9798ef71d14d4088b269ecd998d5783c. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.896983 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:08.897259 29769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:08.897517 29769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:08.899468 29612 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [term 1 FOLLOWER]: Refusing update from remote peer 9798ef71d14d4088b269ecd998d5783c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:08.899700 29536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [term 1 FOLLOWER]: Refusing update from remote peer 9798ef71d14d4088b269ecd998d5783c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:08.900059 29726 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:08.901264 29740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9798ef71d14d4088b269ecd998d5783c. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.901345 29740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:09:08.901463 29726 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:08.902560 29754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9798ef71d14d4088b269ecd998d5783c. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.902643 29754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:09:08.904064 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.904143 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:08.904619 29754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.904695 29754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:09:08.904966 29770 mvcc.cc:205] Tried to move back new op lower bound from 7334448942689693696 to 7334448942680715264. Current Snapshot: MvccSnapshot[applied={T|T < 7334448942689693696}]
I20260928 23:09:08.905169 29773 mvcc.cc:205] Tried to move back new op lower bound from 7334448942689693696 to 7334448942680715264. Current Snapshot: MvccSnapshot[applied={T|T < 7334448942689693696}]
I20260928 23:09:08.905980 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.906060 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:08.906425 29740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9798ef71d14d4088b269ecd998d5783c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7003acc76bb149e68ce2f3b47efbce3b" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45005 } } peers { permanent_uuid: "c07190802cc24ec1b83f55c8f84fbf98" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 36299 } } peers { permanent_uuid: "9798ef71d14d4088b269ecd998d5783c" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 40615 } } }
I20260928 23:09:08.906504 29740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:09:08.907123 29771 mvcc.cc:205] Tried to move back new op lower bound from 7334448942689693696 to 7334448942680715264. Current Snapshot: MvccSnapshot[applied={T|T < 7334448942689693696}]
I20260928 23:09:08.907636 29769 catalog_manager.cc:1347] Generated new cluster ID: e061bd9801164c1a8d4c6b2b9e1e2979
I20260928 23:09:08.907678 29769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:08.921351 29769 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:09:08.922111 29769 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:08.930665 29769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9798ef71d14d4088b269ecd998d5783c: Generated new TSK 0
I20260928 23:09:08.930925 29769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:09:09.820420 29751 catalog_manager.cc:1259] Loaded cluster ID: e061bd9801164c1a8d4c6b2b9e1e2979
I20260928 23:09:09.820556 29751 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: loading cluster ID for follower catalog manager: success
I20260928 23:09:09.821344 29751 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: acquiring CA information for follower catalog manager: success
I20260928 23:09:09.821657 29751 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7003acc76bb149e68ce2f3b47efbce3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:09:09.853281 29765 catalog_manager.cc:1259] Loaded cluster ID: e061bd9801164c1a8d4c6b2b9e1e2979
I20260928 23:09:09.853335 29765 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: loading cluster ID for follower catalog manager: success
I20260928 23:09:09.854139 29765 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: acquiring CA information for follower catalog manager: success
I20260928 23:09:09.854537 29765 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c07190802cc24ec1b83f55c8f84fbf98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:09:09.991288 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29495
I20260928 23:09:09.996719 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29570
I20260928 23:09:10.001623 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29645
2026-09-28T23:09:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2805 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:09:10 dist-test-slave-x2t8 krb5kdc[29787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:09:10 dist-test-slave-x2t8 krb5kdc[29787](info): set up 2 sockets
Sep 28 23:09:10 dist-test-slave-x2t8 krb5kdc[29787](info): commencing operation
krb5kdc: starting...
W20260928 23:09:11.040411 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:09:11.151744 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45211
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:11.230530 29800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:11.230728 29800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:11.230751 29800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:11.231899 29800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:11.231941 29800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:11.231956 29800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:11.231992 29800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:11.232019 29800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:11.233461 29800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:11.233702 29800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:11.233924 29800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:11.238691 29805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:11.238693 29806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:11.238708 29808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:11.239008 29800 server_base.cc:1034] running on GCE node
I20260928 23:09:11.239243 29800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:11.239467 29800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:11.240633 29800 hybrid_clock.cc:648] HybridClock initialized: now 1790636951240623 us; error 37 us; skew 500 ppm
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636951, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:11.250465 29800 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:11.250890 29800 webserver.cc:460] Webserver started at http://127.15.212.126:45153/ using document root <none> and password file <none>
I20260928 23:09:11.251117 29800 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:11.251183 29800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:11.251335 29800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:11.252367 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "bbc17332acc74f8a90333dac9acf8bb3"
format_stamp: "Formatted at 2026-09-28 23:09:11 on dist-test-slave-x2t8"
server_key: "f89b842203c507294ec8b83815d2bc16"
server_key_iv: "718b412c2a551f63376d8e7fb79d28f3"
server_key_version: "encryptionkey@0"
I20260928 23:09:11.252790 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "bbc17332acc74f8a90333dac9acf8bb3"
format_stamp: "Formatted at 2026-09-28 23:09:11 on dist-test-slave-x2t8"
server_key: "f89b842203c507294ec8b83815d2bc16"
server_key_iv: "718b412c2a551f63376d8e7fb79d28f3"
server_key_version: "encryptionkey@0"
I20260928 23:09:11.254077 29800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:11.254791 29815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:11.254954 29800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:11.255069 29800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "bbc17332acc74f8a90333dac9acf8bb3"
format_stamp: "Formatted at 2026-09-28 23:09:11 on dist-test-slave-x2t8"
server_key: "f89b842203c507294ec8b83815d2bc16"
server_key_iv: "718b412c2a551f63376d8e7fb79d28f3"
server_key_version: "encryptionkey@0"
I20260928 23:09:11.255151 29800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:11.267189 29800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:11.268880 29800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:11.269038 29800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:11.272596 29800 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:45211
I20260928 23:09:11.272646 29867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:45211 every 8 connection(s)
I20260928 23:09:11.272975 29800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:11.273764 29868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:11.275802 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29800
I20260928 23:09:11.275820 29868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3: Bootstrap starting.
I20260928 23:09:11.275919 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:09:11.276108 16209 external_mini_cluster.cc:1428] Setting key d2b1ae0829ef2d0364e292123ff8963c
I20260928 23:09:11.276499 29868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:11.276856 29868 log.cc:824] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:11.277594 29868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3: No bootstrap required, opened a new log
I20260928 23:09:11.279747 29868 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } }
I20260928 23:09:11.279903 29868 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:11.279934 29868 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc17332acc74f8a90333dac9acf8bb3, State: Initialized, Role: FOLLOWER
I20260928 23:09:11.280045 29868 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } }
I20260928 23:09:11.280133 29868 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:11.280179 29868 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:11.280233 29868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:11.280848 29868 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } }
I20260928 23:09:11.280968 29868 leader_election.cc:302] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbc17332acc74f8a90333dac9acf8bb3; no voters:
I20260928 23:09:11.281145 29868 leader_election.cc:288] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:09:11.281179 29873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:11.281327 29873 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [term 1 LEADER]: Becoming Leader. State: Replica: bbc17332acc74f8a90333dac9acf8bb3, State: Running, Role: LEADER
I20260928 23:09:11.281405 29868 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:11.281454 29873 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } }
I20260928 23:09:11.281874 29875 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbc17332acc74f8a90333dac9acf8bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } } }
I20260928 23:09:11.281867 29873 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbc17332acc74f8a90333dac9acf8bb3. Latest consensus state: current_term: 1 leader_uuid: "bbc17332acc74f8a90333dac9acf8bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc17332acc74f8a90333dac9acf8bb3" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45211 } } }
I20260928 23:09:11.281956 29875 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:11.281965 29873 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:11.282143 29879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:11.282636 29879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:11.284304 29879 catalog_manager.cc:1347] Generated new cluster ID: cdc8365cd1e5454795766ce2d2feb6f3
I20260928 23:09:11.284360 29879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:11.287051 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:11.276614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.15.212.126:45211)
0928 23:09:11.276785 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:11.276792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:11.276816 (+ 24us) server_negotiation.cc:407] Connection header received
0928 23:09:11.277270 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:11.277276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:11.277321 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:11.277448 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:11.278182 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:11.279003 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:11.280166 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:11.280740 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:11.280826 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:11.283736 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:11.283740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:11.283740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:11.283751 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:11.285505 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:11.285893 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:11.285896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:11.285896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:11.285991 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:11.286205 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:11.286206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:11.286206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:11.286434 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:11.286603 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:11.286713 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 23:09:11.286850 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260928 23:09:11.287935 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:09:11.295492 29879 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:09:11.295986 29879 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:11.298659 29879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbc17332acc74f8a90333dac9acf8bb3: Generated new TSK 0
I20260928 23:09:11.298822 29879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:09:11.341742 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29800
2026-09-28T23:09:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1338 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29895](info): set up 2 sockets
Sep 28 23:09:11 dist-test-slave-x2t8 krb5kdc[29895](info): commencing operation
krb5kdc: starting...
W20260928 23:09:12.380579 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:09:12.493498 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:36273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:12.571522 29908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:12.571718 29908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:12.571740 29908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:12.572805 29908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:12.572837 29908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:12.572850 29908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:12.572862 29908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:12.572875 29908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:12.574234 29908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:36273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:12.574412 29908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:12.574566 29908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:12.579264 29913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.579262 29914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.579257 29916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:12.579470 29908 server_base.cc:1034] running on GCE node
I20260928 23:09:12.579717 29908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:12.580015 29908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:12.581187 29908 hybrid_clock.cc:648] HybridClock initialized: now 1790636952581182 us; error 39 us; skew 500 ppm
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:12.583635 29908 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:12.584095 29908 webserver.cc:460] Webserver started at http://127.15.212.126:45749/ using document root <none> and password file <none>
I20260928 23:09:12.584332 29908 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:12.584399 29908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:12.584553 29908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:12.585594 29908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "00ea5f9ff60c4a42ac1425b697ef5431"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "b2f323103481f51188ede7be9db79143"
server_key_iv: "0abb802d36dafced75f7c56158373818"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.585966 29908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "00ea5f9ff60c4a42ac1425b697ef5431"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "b2f323103481f51188ede7be9db79143"
server_key_iv: "0abb802d36dafced75f7c56158373818"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.587240 29908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:09:12.587952 29923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.588127 29908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:09:12.588225 29908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "00ea5f9ff60c4a42ac1425b697ef5431"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "b2f323103481f51188ede7be9db79143"
server_key_iv: "0abb802d36dafced75f7c56158373818"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.588312 29908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:12.606369 29908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:12.608796 29908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:12.608961 29908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:12.612910 29908 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:36273
I20260928 23:09:12.612962 29975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:36273 every 8 connection(s)
I20260928 23:09:12.613330 29908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:12.614283 29976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:12.616536 29976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap starting.
I20260928 23:09:12.617113 29976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:12.617388 29976 log.cc:824] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:12.618152 29976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: No bootstrap required, opened a new log
I20260928 23:09:12.618157 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 29908
I20260928 23:09:12.618259 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:09:12.618469 16209 external_mini_cluster.cc:1428] Setting key 98d9093a1eabdf3ba2c7cd94b79dbb69
I20260928 23:09:12.620116 29976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:12.620288 29976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:12.620327 29976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Initialized, Role: FOLLOWER
I20260928 23:09:12.620538 29976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:12.620638 29976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:12.620674 29976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:12.620729 29976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:12.621682 29976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:12.621805 29976 leader_election.cc:302] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ea5f9ff60c4a42ac1425b697ef5431; no voters:
I20260928 23:09:12.621965 29976 leader_election.cc:288] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:09:12.621996 29981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:12.622175 29981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 LEADER]: Becoming Leader. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Running, Role: LEADER
I20260928 23:09:12.622231 29976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:12.622304 29981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:12.622732 29982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:12.622803 29982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:12.623018 29983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ea5f9ff60c4a42ac1425b697ef5431. Latest consensus state: current_term: 1 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:12.623116 29983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:12.623430 29989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:12.623994 29989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:12.625782 29989 catalog_manager.cc:1347] Generated new cluster ID: 807f73317a994990ac5bf38f03d3fb40
I20260928 23:09:12.625833 29989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:12.630769 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:12.619004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.15.212.126:36273)
0928 23:09:12.619188 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:12.619197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:12.619232 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:09:12.620089 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:12.620095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:12.620152 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:12.620267 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:12.621122 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.622102 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.623242 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.623885 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.623992 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:12.626805 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:12.626809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:12.626809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:12.626822 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:12.628877 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.629245 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.629248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.629248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.629384 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.629582 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.629583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.629584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.629912 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:12.630080 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:12.630156 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 23:09:12.630404 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260928 23:09:12.637893 29989 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:09:12.638631 29989 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:12.652212 29989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Generated new TSK 0
I20260928 23:09:12.652400 29989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:09:12.678653 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:12.758765 30002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:12.758980 30002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:12.759001 30002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:12.760180 30002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:12.760233 30002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:12.760259 30002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:12.761684 30002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:12.761904 30002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:12.762130 30002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:12.766892 30007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.766888 30008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.766907 30010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:12.767043 30002 server_base.cc:1034] running on GCE node
I20260928 23:09:12.767347 30002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:12.767536 30002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:12.768692 30002 hybrid_clock.cc:648] HybridClock initialized: now 1790636952768669 us; error 35 us; skew 500 ppm
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:12.771027 30002 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:12.771422 30002 webserver.cc:460] Webserver started at http://127.15.212.65:45439/ using document root <none> and password file <none>
I20260928 23:09:12.771636 30002 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:12.771680 30002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:12.771839 30002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:12.772854 30002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "d24312d87b6d42fc984a66a8bd68ef06"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "2699ab7ab40443ba5ab3b71581ed31b5"
server_key_iv: "ac3a9b697036434ce8f8319674640697"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.773216 30002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "d24312d87b6d42fc984a66a8bd68ef06"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "2699ab7ab40443ba5ab3b71581ed31b5"
server_key_iv: "ac3a9b697036434ce8f8319674640697"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.774453 30002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:09:12.775122 30017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.775295 30002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:12.775380 30002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d24312d87b6d42fc984a66a8bd68ef06"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "2699ab7ab40443ba5ab3b71581ed31b5"
server_key_iv: "ac3a9b697036434ce8f8319674640697"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.775449 30002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:12.783694 30002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:12.785383 30002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:12.785532 30002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:12.785738 30002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:12.786022 30002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:12.786054 30002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.786075 30002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:12.786123 30002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.791585 30002 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:42213
I20260928 23:09:12.791662 30130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:42213 every 8 connection(s)
I20260928 23:09:12.791983 30002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:12.792913 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30002
I20260928 23:09:12.793040 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:09:12.793234 16209 external_mini_cluster.cc:1428] Setting key 0cb381509e2e699070999d3fabc71b9f
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:12.801506 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:12.792657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:36177 (local address 127.15.212.126:36273)
0928 23:09:12.792729 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:12.792731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:12.792931 (+ 200us) server_negotiation.cc:407] Connection header received
0928 23:09:12.793523 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:12.793524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:12.793537 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:12.793588 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:12.794490 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.794988 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.795963 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.796295 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.796365 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:12.798492 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:12.798493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:12.798493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:12.798506 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:12.800120 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.800547 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.800548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.800549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.800620 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.800855 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.800856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.800856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.801010 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:12.801092 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:12.801325 (+ 233us) server_negotiation.cc:299] Negotiation successful
0928 23:09:12.801410 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
I20260928 23:09:12.801527 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:12.792625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:36177)
0928 23:09:12.792793 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:12.792851 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:12.793417 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:12.793653 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:12.793657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:12.794210 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:12.794396 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:12.794399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.795083 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.795084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:12.795861 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:12.795863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.796384 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.796384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:12.796543 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:12.797036 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:12.797044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:12.798401 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:12.800204 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:12.800210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:12.800211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:12.800414 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:12.800694 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:12.800695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:12.800695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:12.800789 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:12.801087 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:12.801091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:12.801195 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 23:09:12.801272 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 23:09:12.801383 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260928 23:09:12.802132 30131 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:12.802240 30131 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:12.802402 30131 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:12.802953 29940 ts_manager.cc:194] Registered new tserver with Master: d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213)
I20260928 23:09:12.803664 29940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:36177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:09:12.818774 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:12.898013 30136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:12.898228 30136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:12.898263 30136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:12.899395 30136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:12.899463 30136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:12.899507 30136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:12.900933 30136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:12.901196 30136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:12.901407 30136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:12.906174 30144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.906267 30142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:12.906373 30141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:12.906266 30136 server_base.cc:1034] running on GCE node
I20260928 23:09:12.906610 30136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:12.906853 30136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:12.908035 30136 hybrid_clock.cc:648] HybridClock initialized: now 1790636952908024 us; error 30 us; skew 500 ppm
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:12.910424 30136 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:12.910764 30136 webserver.cc:460] Webserver started at http://127.15.212.66:34139/ using document root <none> and password file <none>
I20260928 23:09:12.910924 30136 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:12.910964 30136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:12.911070 30136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:12.911978 30136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "34dd5543913947ef9e350b62fddc84b7"
server_key_iv: "541254a90ba18192881e2a85e9e4c248"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.912272 30136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "34dd5543913947ef9e350b62fddc84b7"
server_key_iv: "541254a90ba18192881e2a85e9e4c248"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.913599 30136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:09:12.914198 30151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.914376 30136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:12.914460 30136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "34dd5543913947ef9e350b62fddc84b7"
server_key_iv: "541254a90ba18192881e2a85e9e4c248"
server_key_version: "encryptionkey@0"
I20260928 23:09:12.914527 30136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:12.948928 30136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:12.951262 30136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:12.951416 30136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:12.951671 30136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:12.951982 30136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:12.952014 30136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.952037 30136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:12.952085 30136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:12.957979 30136 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35733
I20260928 23:09:12.958069 30264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35733 every 8 connection(s)
I20260928 23:09:12.958369 30136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:12.963423 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30136
I20260928 23:09:12.963543 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:09:12.963721 16209 external_mini_cluster.cc:1428] Setting key 1ef77f69bb136dc5b41f2148d7f6ae9d
Sep 28 23:09:12 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636952, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:12.967795 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:12.959085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:47817)
0928 23:09:12.959224 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:12.959259 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:12.959718 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:12.959915 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:12.959918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:12.960551 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:12.960748 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:12.960754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.961393 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.961393 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:12.962181 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:12.962182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.962506 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.962506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:12.962603 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:12.963023 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:12.963031 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:12.964644 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:12.966373 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:12.966380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:12.966382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:12.966704 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:12.966896 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:12.966896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:12.966897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:12.967039 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:12.967322 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:12.967328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:12.967437 (+ 109us) client_negotiation.cc:769] Sending connection context
0928 23:09:12.967521 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 23:09:12.967640 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260928 23:09:12.967803 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:12.959103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:47817 (local address 127.15.212.126:36273)
0928 23:09:12.959154 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:12.959156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:12.959354 (+ 198us) server_negotiation.cc:407] Connection header received
0928 23:09:12.959799 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:12.959800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:12.959810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:12.959863 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:12.960824 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.961314 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.962252 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:12.962435 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:12.962486 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:12.964728 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:12.964730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:12.964730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:12.964738 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:12.966292 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.966783 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.966784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.966784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.966833 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:12.967105 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:12.967105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:12.967106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:12.967246 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:12.967313 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:12.967573 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 23:09:12.967665 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 23:09:12.968241 30265 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:12.968351 30265 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:12.968564 30265 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:12.968933 29940 ts_manager.cc:194] Registered new tserver with Master: 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
I20260928 23:09:12.969259 29940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:47817
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:09:12.988545 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:13.067049 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:13.067229 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:13.067250 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:13.068374 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:13.068414 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:13.068462 30270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:13.069900 30270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:13.070132 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:13.070349 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:13.075296 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.075412 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.075376 30278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:13.075562 30270 server_base.cc:1034] running on GCE node
I20260928 23:09:13.075768 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:13.075990 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:13.077155 30270 hybrid_clock.cc:648] HybridClock initialized: now 1790636953077146 us; error 30 us; skew 500 ppm
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:13.079582 30270 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:13.080017 30270 webserver.cc:460] Webserver started at http://127.15.212.67:44207/ using document root <none> and password file <none>
I20260928 23:09:13.080261 30270 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:13.080304 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:13.080489 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:13.081506 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "9e813ea75d5e4c02af0b74f945c520d9"
format_stamp: "Formatted at 2026-09-28 23:09:13 on dist-test-slave-x2t8"
server_key: "8d053216346509b6f3b901963aaf377b"
server_key_iv: "7c92fb7427128d4251da11e8a40d6f82"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.081843 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "9e813ea75d5e4c02af0b74f945c520d9"
format_stamp: "Formatted at 2026-09-28 23:09:13 on dist-test-slave-x2t8"
server_key: "8d053216346509b6f3b901963aaf377b"
server_key_iv: "7c92fb7427128d4251da11e8a40d6f82"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.083091 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:09:13.083766 30285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.083958 30270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:09:13.084038 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "9e813ea75d5e4c02af0b74f945c520d9"
format_stamp: "Formatted at 2026-09-28 23:09:13 on dist-test-slave-x2t8"
server_key: "8d053216346509b6f3b901963aaf377b"
server_key_iv: "7c92fb7427128d4251da11e8a40d6f82"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.084087 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:13.109393 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:13.111238 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:13.111340 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:13.111505 30270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:13.111800 30270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:13.111835 30270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.111856 30270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:13.111889 30270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.117561 30270 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41545
I20260928 23:09:13.117609 30398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41545 every 8 connection(s)
I20260928 23:09:13.117899 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:09:13.122820 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30270
I20260928 23:09:13.122934 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:09:13.123100 16209 external_mini_cluster.cc:1428] Setting key a72f183c1e4f239cd9932bbc10851d51
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:13.127381 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.118559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:38889 (local address 127.15.212.126:36273)
0928 23:09:13.118599 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.118601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.118813 (+ 212us) server_negotiation.cc:407] Connection header received
0928 23:09:13.119262 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.119263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.119272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.119326 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.120193 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.120677 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.121600 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.121848 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.121900 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.124871 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.124872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.124873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.124881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.126028 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.126433 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.126434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.126435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.126496 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.126772 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.126773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.126773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.126886 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.126958 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.127216 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.127296 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:09:13.127549 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.118530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:38889)
0928 23:09:13.118690 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.118729 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.119186 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.119393 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.119396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.119949 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:13.120120 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.120124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.120751 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.120751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.121527 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.121529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.121919 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.121921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.122018 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.122505 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:13.122512 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:13.124777 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:13.126115 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.126121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.126123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.126341 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.126590 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.126590 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.126591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.126697 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.126949 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:13.126955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:13.127090 (+ 135us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.127193 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.127334 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260928 23:09:13.127939 30399 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:13.128021 30399 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:13.128145 30399 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:13.128477 29940 ts_manager.cc:194] Registered new tserver with Master: 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545)
I20260928 23:09:13.128799 29940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:38889
I20260928 23:09:13.134871 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:13.141906 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.136038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.15.212.126:36273)
0928 23:09:13.136117 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.136119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.136241 (+ 122us) server_negotiation.cc:407] Connection header received
0928 23:09:13.136315 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.136315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.136325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.136371 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.136922 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.137368 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.138127 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.138315 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.138388 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.139496 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.139497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.139498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.139505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.140760 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.141114 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.141115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.141115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.141170 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.141376 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.141377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.141377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.141513 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.141633 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.141734 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.141808 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 23:09:13.143147 29940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:09:13.144651 29940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:09:13.150995 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.146977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.15.212.66:35733)
0928 23:09:13.147230 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.147235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.147268 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:09:13.147368 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.147372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.147391 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.147469 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.147745 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.148546 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.149829 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.150549 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150634 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.150688 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.150873 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.150912 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260928 23:09:13.151015 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.146869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35733 (local address 127.0.0.1:36592)
0928 23:09:13.147172 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.147215 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.147287 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.147593 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.147596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.147598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.147661 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.147663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.148642 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.148643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.149738 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.149741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.150666 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.150750 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.150764 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.150853 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.150909 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":73,"threads_started":1}
I20260928 23:09:13.151213 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.147077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41545 (local address 127.0.0.1:56792)
0928 23:09:13.147398 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.147407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.147455 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.147690 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.147691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.147692 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.147838 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.147839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.149080 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.149081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.150100 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.150103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.150944 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.151020 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.151030 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.151083 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.151136 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":111,"thread_start_us":34,"threads_started":1}
I20260928 23:09:13.151548 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.147162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.15.212.67:41545)
0928 23:09:13.147318 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.147322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.147515 (+ 193us) server_negotiation.cc:407] Connection header received
0928 23:09:13.147555 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.147557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.147572 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.147636 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.147908 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.148972 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150215 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.150799 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150861 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.150911 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.151396 (+ 485us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.151468 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260928 23:09:13.152077 30199 tablet_service.cc:1467] Processing CreateTablet for tablet 6861a068983447beb236c1bc19ccf84c (DEFAULT_TABLE table=test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.152135 30333 tablet_service.cc:1467] Processing CreateTablet for tablet 6861a068983447beb236c1bc19ccf84c (DEFAULT_TABLE table=test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.152474 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.146914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:42213 (local address 127.0.0.1:49446)
0928 23:09:13.147362 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.147372 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.147437 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.147682 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.147683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.147685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.147758 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.147759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.150187 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.150188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.151249 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.151251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.152224 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.152225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.152282 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.152286 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.152336 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.152371 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260928 23:09:13.152490 30333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6861a068983447beb236c1bc19ccf84c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.152513 30199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6861a068983447beb236c1bc19ccf84c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.152740 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.147162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.15.212.65:42213)
0928 23:09:13.147359 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.147363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.147470 (+ 107us) server_negotiation.cc:407] Connection header received
0928 23:09:13.147513 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.147516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.147532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.147607 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.147824 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.148964 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.151340 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.152032 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.152124 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.152189 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.152625 (+ 436us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.152668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
I20260928 23:09:13.153330 30065 tablet_service.cc:1467] Processing CreateTablet for tablet 6861a068983447beb236c1bc19ccf84c (DEFAULT_TABLE table=test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.153643 30065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6861a068983447beb236c1bc19ccf84c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.154934 30413 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Bootstrap starting.
I20260928 23:09:13.155594 30413 tablet_bootstrap.cc:654] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.155874 30414 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Bootstrap starting.
I20260928 23:09:13.156145 30413 log.cc:824] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.156558 30414 tablet_bootstrap.cc:654] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.156836 30416 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap starting.
I20260928 23:09:13.157065 30414 log.cc:824] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.157164 30413 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: No bootstrap required, opened a new log
I20260928 23:09:13.157262 30413 ts_tablet_manager.cc:1397] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:09:13.157501 30416 tablet_bootstrap.cc:654] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.157915 30414 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: No bootstrap required, opened a new log
I20260928 23:09:13.157989 30414 ts_tablet_manager.cc:1397] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:09:13.158049 30416 log.cc:824] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.158720 30416 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: No bootstrap required, opened a new log
I20260928 23:09:13.158794 30416 ts_tablet_manager.cc:1397] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:09:13.159181 30413 raft_consensus.cc:348] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.159343 30413 raft_consensus.cc:374] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.159386 30413 raft_consensus.cc:729] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e813ea75d5e4c02af0b74f945c520d9, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.159540 30413 consensus_queue.cc:260] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.159807 30413 ts_tablet_manager.cc:1428] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:09:13.159945 30399 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
I20260928 23:09:13.160001 30414 raft_consensus.cc:348] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.160127 30414 raft_consensus.cc:374] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.160147 30414 raft_consensus.cc:729] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a4a2cf1059046a3bd0e3d2825cc6aa2, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.160248 30414 consensus_queue.cc:260] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.160526 30265 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
I20260928 23:09:13.160568 30416 raft_consensus.cc:348] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.160513 30414 ts_tablet_manager.cc:1428] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:09:13.160750 30416 raft_consensus.cc:374] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.160804 30416 raft_consensus.cc:729] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24312d87b6d42fc984a66a8bd68ef06, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.160921 30416 consensus_queue.cc:260] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.161298 30416 ts_tablet_manager.cc:1428] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:09:13.161350 30131 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
W20260928 23:09:13.208932 30266 tablet.cc:2369] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:09:13.292742 30132 tablet.cc:2369] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:13.335631 30420 raft_consensus.cc:482] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:13.335755 30420 raft_consensus.cc:504] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.336192 30420 leader_election.cc:288] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213), 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545)
I20260928 23:09:13.339170 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.336402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:42213 (local address 127.15.212.66:37557)
0928 23:09:13.336470 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.336486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.336545 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.336704 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.336705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.336707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.336792 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.336794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.337554 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.337554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.338361 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.338363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.338919 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.338920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.339002 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.339006 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.339062 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.339105 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":5}
I20260928 23:09:13.339210 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.336504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:37557 (local address 127.15.212.65:42213)
0928 23:09:13.336572 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.336574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.336586 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:09:13.336615 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.336616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.336627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.336696 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.336871 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.337478 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.338448 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.338846 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.338900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.338906 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.339083 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.339120 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:09:13.339331 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.336611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41545 (local address 127.15.212.66:51265)
0928 23:09:13.336764 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.336773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.336825 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.336978 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.336979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.336980 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.337052 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.337054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.337741 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.337742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.338480 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.338482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.339073 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.339074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.339142 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.339146 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.339204 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.339244 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260928 23:09:13.339354 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.336704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:51265 (local address 127.15.212.67:41545)
0928 23:09:13.336755 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.336757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.336819 (+ 62us) server_negotiation.cc:407] Connection header received
0928 23:09:13.336898 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.336899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.336907 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.336956 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.337127 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.337661 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.338564 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.338971 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.339047 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.339055 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.339223 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.339268 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 23:09:13.339574 30085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6861a068983447beb236c1bc19ccf84c" candidate_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24312d87b6d42fc984a66a8bd68ef06" is_pre_election: true
I20260928 23:09:13.339730 30085 raft_consensus.cc:2454] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a4a2cf1059046a3bd0e3d2825cc6aa2 in term 0.
I20260928 23:09:13.339653 30353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6861a068983447beb236c1bc19ccf84c" candidate_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e813ea75d5e4c02af0b74f945c520d9" is_pre_election: true
I20260928 23:09:13.339825 30353 raft_consensus.cc:2454] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a4a2cf1059046a3bd0e3d2825cc6aa2 in term 0.
I20260928 23:09:13.340066 30152 leader_election.cc:302] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9; no voters:
I20260928 23:09:13.340188 30420 raft_consensus.cc:2781] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:13.340243 30420 raft_consensus.cc:482] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:13.340266 30420 raft_consensus.cc:3037] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:13.340952 30420 raft_consensus.cc:504] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.341104 30420 leader_election.cc:288] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [CANDIDATE]: Term 1 election: Requested vote from peers d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213), 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545)
I20260928 23:09:13.341274 30085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6861a068983447beb236c1bc19ccf84c" candidate_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24312d87b6d42fc984a66a8bd68ef06"
I20260928 23:09:13.341313 30353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6861a068983447beb236c1bc19ccf84c" candidate_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e813ea75d5e4c02af0b74f945c520d9"
I20260928 23:09:13.341375 30353 raft_consensus.cc:3037] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:13.341382 30085 raft_consensus.cc:3037] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:13.342065 30353 raft_consensus.cc:2454] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a4a2cf1059046a3bd0e3d2825cc6aa2 in term 1.
I20260928 23:09:13.342224 30152 leader_election.cc:302] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9; no voters:
I20260928 23:09:13.342288 30085 raft_consensus.cc:2454] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a4a2cf1059046a3bd0e3d2825cc6aa2 in term 1.
I20260928 23:09:13.342319 30420 raft_consensus.cc:2781] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:13.342481 30420 raft_consensus.cc:686] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 LEADER]: Becoming Leader. State: Replica: 3a4a2cf1059046a3bd0e3d2825cc6aa2, State: Running, Role: LEADER
I20260928 23:09:13.342610 30420 consensus_queue.cc:237] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.343443 29939 catalog_manager.cc:5495] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66). New cstate: current_term: 1 leader_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } health_report { overall_health: UNKNOWN } } }
W20260928 23:09:13.368474 30400 tablet.cc:2369] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:13.378427 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.376122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.15.212.66:35733)
0928 23:09:13.376186 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.376188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.376213 (+ 25us) server_negotiation.cc:407] Connection header received
0928 23:09:13.376256 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.376257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.376273 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.376325 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:13.376476 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.376963 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.377706 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.377918 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.377965 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.378103 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:13.378203 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:13.378259 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.378293 (+ 34us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.378308 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:09:13.380192 30353 raft_consensus.cc:1261] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Refusing update from remote peer 3a4a2cf1059046a3bd0e3d2825cc6aa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:13.380193 30085 raft_consensus.cc:1261] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Refusing update from remote peer 3a4a2cf1059046a3bd0e3d2825cc6aa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:13.380636 30423 consensus_queue.cc:1035] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:13.380769 30420 consensus_queue.cc:1035] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:13.383078 30426 mvcc.cc:205] Tried to move back new op lower bound from 7334448961042563072 to 7334448960891858944. Current Snapshot: MvccSnapshot[applied={T|T < 7334448961042563072}]
I20260928 23:09:13.383212 30427 mvcc.cc:205] Tried to move back new op lower bound from 7334448961042563072 to 7334448960891858944. Current Snapshot: MvccSnapshot[applied={T|T < 7334448961042563072}]
I20260928 23:09:13.383733 30428 mvcc.cc:205] Tried to move back new op lower bound from 7334448961042563072 to 7334448960891858944. Current Snapshot: MvccSnapshot[applied={T|T < 7334448961042563072}]
I20260928 23:09:13.385828 29939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:13.385915 29939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:13.386601 29939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Sending DeleteTablet for 3 replicas of tablet 6861a068983447beb236c1bc19ccf84c
I20260928 23:09:13.386950 30199 tablet_service.cc:1514] Processing DeleteTablet for tablet 6861a068983447beb236c1bc19ccf84c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:13 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36592
I20260928 23:09:13.387001 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30002
I20260928 23:09:13.387629 30333 tablet_service.cc:1514] Processing DeleteTablet for tablet 6861a068983447beb236c1bc19ccf84c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:13 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:56792
I20260928 23:09:13.388090 30436 tablet_replica.cc:330] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: stopping tablet replica
I20260928 23:09:13.388201 30436 raft_consensus.cc:2229] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:09:13.388294 30436 raft_consensus.cc:2258] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:13.388746 30436 ts_tablet_manager.cc:1905] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:13.390681 30436 ts_tablet_manager.cc:1918] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:13.390790 30436 log.cc:1196] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/6861a068983447beb236c1bc19ccf84c
I20260928 23:09:13.391156 30436 ts_tablet_manager.cc:1939] T 6861a068983447beb236c1bc19ccf84c P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting consensus metadata
I20260928 23:09:13.391569 29927 catalog_manager.cc:4841] TS 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): tablet 6861a068983447beb236c1bc19ccf84c (table test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]) successfully deleted
W20260928 23:09:13.392333 29924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:42213 (error 108)
I20260928 23:09:13.392762 30410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.392632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:42213 (local address 127.0.0.1:49454)
0928 23:09:13.392688 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.392724 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:09:13.392863 29924 catalog_manager.cc:4584] TS d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6861a068983447beb236c1bc19ccf84c: Network error: Client connection negotiation failed: client connection to 127.15.212.65:42213: connect: Connection refused (error 111)
I20260928 23:09:13.393148 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30136
I20260928 23:09:13.394729 30437 tablet_replica.cc:330] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: stopping tablet replica
I20260928 23:09:13.394865 30437 raft_consensus.cc:2229] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:13.394989 30437 raft_consensus.cc:2258] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:13.395387 30437 ts_tablet_manager.cc:1905] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:13.396729 30437 ts_tablet_manager.cc:1918] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:13.396788 30437 log.cc:1196] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/6861a068983447beb236c1bc19ccf84c
I20260928 23:09:13.397100 30437 ts_tablet_manager.cc:1939] T 6861a068983447beb236c1bc19ccf84c P 9e813ea75d5e4c02af0b74f945c520d9: Deleting consensus metadata
I20260928 23:09:13.397481 29924 catalog_manager.cc:4841] TS 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): tablet 6861a068983447beb236c1bc19ccf84c (table test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]) successfully deleted
I20260928 23:09:13.398474 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30270
I20260928 23:09:13.405311 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 29908
I20260928 23:09:13.410509 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--webserver_interface=127.15.212.126
--webserver_port=45749
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:36273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:13.489465 30439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:13.489708 30439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:13.489748 30439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:13.490869 30439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:13.490938 30439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:13.490962 30439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:13.490988 30439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:13.491039 30439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:13.492489 30439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:36273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=45749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:13.492756 30439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:13.492969 30439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:13.501257 30445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.501287 30446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.501257 30448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:13.501550 30439 server_base.cc:1034] running on GCE node
I20260928 23:09:13.501833 30439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:13.502157 30439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:13.503324 30439 hybrid_clock.cc:648] HybridClock initialized: now 1790636953503303 us; error 34 us; skew 500 ppm
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:13.505789 30439 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:13.506194 30439 webserver.cc:460] Webserver started at http://127.15.212.126:45749/ using document root <none> and password file <none>
I20260928 23:09:13.506428 30439 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:13.506506 30439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:13.507777 30439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:13.508385 30455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.508580 30439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:13.508683 30439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "00ea5f9ff60c4a42ac1425b697ef5431"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "b2f323103481f51188ede7be9db79143"
server_key_iv: "0abb802d36dafced75f7c56158373818"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.509037 30439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:13.517160 30439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:13.518785 30439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:13.518930 30439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:13.522526 30439 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:36273
I20260928 23:09:13.522557 30507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:36273 every 8 connection(s)
I20260928 23:09:13.522907 30439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:13.524349 30508 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:09:13.524993 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30439
I20260928 23:09:13.525022 30508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap starting.
I20260928 23:09:13.525270 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:42213
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=45439
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:13.533500 30508 log.cc:824] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.535348 30508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:13.535727 30508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap complete.
I20260928 23:09:13.537954 30508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:13.538197 30508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.538394 30508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:13.538488 30508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:13.538545 30508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:13.538594 30508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:09:13.539546 30508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:13.539690 30508 leader_election.cc:302] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ea5f9ff60c4a42ac1425b697ef5431; no voters:
I20260928 23:09:13.539841 30508 leader_election.cc:288] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 23:09:13.539875 30512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 23:09:13.539981 30512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 LEADER]: Becoming Leader. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Running, Role: LEADER
I20260928 23:09:13.540092 30512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:13.540210 30508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:13.540359 30512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:13.540541 30512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:13.540690 30512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ea5f9ff60c4a42ac1425b697ef5431. Latest consensus state: current_term: 2 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:13.540772 30517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:13.540838 30512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:13.541788 30517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:13.541939 30517 catalog_manager.cc:1259] Loaded cluster ID: 807f73317a994990ac5bf38f03d3fb40
I20260928 23:09:13.542212 30517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:13.542479 30517 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 23:09:13.547096 30517 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 23:09:13.549391 30517 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:13.549561 30517 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Loaded TSK: 0
I20260928 23:09:13.549788 30517 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 23:09:13.606725 30510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:13.606951 30510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:13.607004 30510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:13.608151 30510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:13.608215 30510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:13.608259 30510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:13.609755 30510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:42213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=45439
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:13.609990 30510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:13.610209 30510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:13.615989 30535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.616426 30536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.616513 30538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:13.616748 30510 server_base.cc:1034] running on GCE node
I20260928 23:09:13.616942 30510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:13.617173 30510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:13.618332 30510 hybrid_clock.cc:648] HybridClock initialized: now 1790636953618320 us; error 29 us; skew 500 ppm
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:13.620935 30510 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:13.621428 30510 webserver.cc:460] Webserver started at http://127.15.212.65:45439/ using document root <none> and password file <none>
I20260928 23:09:13.621644 30510 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:13.621724 30510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:13.623082 30510 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:13.623759 30545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.623929 30510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:13.624027 30510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d24312d87b6d42fc984a66a8bd68ef06"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "2699ab7ab40443ba5ab3b71581ed31b5"
server_key_iv: "ac3a9b697036434ce8f8319674640697"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.624354 30510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:13.648813 30510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:13.650892 30510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:13.651027 30510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:13.651305 30510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:13.651816 30552 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 23:09:13.653228 30510 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 23:09:13.653277 30510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 23:09:13.653311 30510 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 23:09:13.653947 30510 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 23:09:13.653990 30510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:13.654031 30552 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap starting.
I20260928 23:09:13.661571 30510 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:42213
I20260928 23:09:13.661664 30659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:42213 every 8 connection(s)
I20260928 23:09:13.662153 30510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:13.665653 30552 log.cc:824] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.667734 30552 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:13.668009 30552 tablet_bootstrap.cc:492] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap complete.
I20260928 23:09:13.669178 30552 ts_tablet_manager.cc:1397] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:13.670244 30552 raft_consensus.cc:348] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.670560 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30510
I20260928 23:09:13.670553 30552 raft_consensus.cc:729] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24312d87b6d42fc984a66a8bd68ef06, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.670706 30552 consensus_queue.cc:260] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } } peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } }
I20260928 23:09:13.670827 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:35733
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=34139
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:13.670926 30552 ts_tablet_manager.cc:1428] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260928 23:09:13.675004 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.662960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:36495 (local address 127.15.212.126:36273)
0928 23:09:13.663149 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.663158 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.663199 (+ 41us) server_negotiation.cc:407] Connection header received
0928 23:09:13.663950 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.663959 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.664038 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.664208 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.664958 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.666252 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.667711 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.668338 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.668464 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.671621 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.671625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.671625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.671637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.673287 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.673781 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.673783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.673783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.673883 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.674156 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.674157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.674157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.674348 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.674407 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.674779 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.674875 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260928 23:09:13.675132 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.662873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:36495)
0928 23:09:13.663066 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.663109 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.663690 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.664245 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.664249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.664587 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:13.664822 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.664827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.666366 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.666367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.667619 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.667622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.668474 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.668475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.668624 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.669225 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:13.669234 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:13.671484 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:13.673372 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.673378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.673380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.673696 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.673953 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.673954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.673955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.674090 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.674421 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:13.674427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:13.674594 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.674715 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.674896 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260928 23:09:13.675819 30660 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:13.675925 30660 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:13.676082 30660 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:13.676858 30472 ts_manager.cc:194] Registered new tserver with Master: d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213)
I20260928 23:09:13.678143 30472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:36495
I20260928 23:09:13.679039 30660 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:09:13.686309 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.677491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:42213 (local address 127.0.0.1:49468)
0928 23:09:13.677650 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.677682 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.677740 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.678725 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.678731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.679076 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:13.679134 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.679136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.680082 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.680083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.681145 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.681148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.681569 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.681571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.681659 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.682064 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:13.682070 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:13.683411 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:13.685186 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.685192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.685194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.685385 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.685612 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.685613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.685613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.685672 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.685947 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:13.685953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:13.686002 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.686103 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.686219 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260928 23:09:13.686308 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.678429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.15.212.65:42213)
0928 23:09:13.678572 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.678576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.678584 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:13.678609 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.678610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.678630 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.678718 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.679224 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.680001 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.681232 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.681488 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.681537 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.683525 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.683532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.683533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.683546 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.685088 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.685482 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.685485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.685485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.685542 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.685744 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.685745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.685745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.685874 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.685965 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.686139 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.686219 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260928 23:09:13.686849 30578 tablet_service.cc:1514] Processing DeleteTablet for tablet 6861a068983447beb236c1bc19ccf84c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:13 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49468
I20260928 23:09:13.687309 30669 tablet_replica.cc:330] stopping tablet replica
I20260928 23:09:13.687420 30669 raft_consensus.cc:2229] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:13.687491 30669 raft_consensus.cc:2258] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:13.687837 30669 ts_tablet_manager.cc:1905] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:13.690194 30669 ts_tablet_manager.cc:1918] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:13.690263 30669 log.cc:1196] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/6861a068983447beb236c1bc19ccf84c
I20260928 23:09:13.690500 30669 ts_tablet_manager.cc:1939] T 6861a068983447beb236c1bc19ccf84c P d24312d87b6d42fc984a66a8bd68ef06: Deleting consensus metadata
I20260928 23:09:13.690959 30456 catalog_manager.cc:4841] TS d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213): tablet 6861a068983447beb236c1bc19ccf84c (table test-table [id=07e63c0a000e4af6ba721ae2259cdb4c]) successfully deleted
W20260928 23:09:13.755148 30666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:13.755327 30666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:13.755348 30666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:13.756670 30666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:13.756711 30666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:13.756737 30666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:13.758172 30666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:35733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=34139
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:13.758387 30666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:13.758602 30666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:09:13.763716 30666 server_base.cc:1034] running on GCE node
W20260928 23:09:13.763665 30675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.763690 30677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.763679 30674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:13.764035 30666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:13.764281 30666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:13.765415 30666 hybrid_clock.cc:648] HybridClock initialized: now 1790636953765404 us; error 31 us; skew 500 ppm
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:13.767745 30666 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:13.768187 30666 webserver.cc:460] Webserver started at http://127.15.212.66:34139/ using document root <none> and password file <none>
I20260928 23:09:13.768399 30666 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:13.768508 30666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:13.769817 30666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:13.770428 30684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.770602 30666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:13.770689 30666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "34dd5543913947ef9e350b62fddc84b7"
server_key_iv: "541254a90ba18192881e2a85e9e4c248"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.770994 30666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:13.795513 30666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:13.797446 30666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:13.797605 30666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:13.797828 30666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:13.798156 30666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:13.798210 30666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.798247 30666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:13.798273 30666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.803879 30666 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35733
I20260928 23:09:13.803927 30797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35733 every 8 connection(s)
I20260928 23:09:13.804250 30666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:13.806497 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30666
I20260928 23:09:13.806783 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:41545
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=44207
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:13.814298 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.804949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:35247 (local address 127.15.212.126:36273)
0928 23:09:13.805014 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.805016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.805238 (+ 222us) server_negotiation.cc:407] Connection header received
0928 23:09:13.805716 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.805717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.805726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.805781 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.806728 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.807490 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.808840 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.809036 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.809092 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.811408 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.811409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.811409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.811423 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.812936 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.813358 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.813360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.813360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.813418 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.813661 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.813662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.813663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.813773 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.813850 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.814122 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.814207 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:09:13.814481 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.804949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:35247)
0928 23:09:13.805114 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.805154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.805636 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.805832 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.805835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.806400 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:13.806637 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.806643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.807578 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.807579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.808757 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.808761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.809122 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.809123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.809260 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.809727 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:13.809736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:13.811304 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:13.813039 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.813045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.813047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.813269 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.813484 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.813485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.813486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.813594 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.813840 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:13.813845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:13.813979 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.814099 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.814247 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260928 23:09:13.814937 30798 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:13.815042 30798 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:13.815212 30798 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:13.815619 30472 ts_manager.cc:194] Registered new tserver with Master: 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
I20260928 23:09:13.815991 30472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:35247
W20260928 23:09:13.887723 30801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:13.887936 30801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:13.887984 30801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:13.889146 30801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:13.889213 30801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:13.889257 30801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:13.890748 30801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:41545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=44207
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:13.890997 30801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:13.891219 30801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:13.895912 30809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.895977 30806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:13.895895 30807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:13.896064 30801 server_base.cc:1034] running on GCE node
I20260928 23:09:13.896277 30801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:13.896572 30801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:13.897753 30801 hybrid_clock.cc:648] HybridClock initialized: now 1790636953897743 us; error 37 us; skew 500 ppm
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:13.900051 30801 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:13.900520 30801 webserver.cc:460] Webserver started at http://127.15.212.67:44207/ using document root <none> and password file <none>
I20260928 23:09:13.900748 30801 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:13.900827 30801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:13.902079 30801 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:13.902602 30816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.902762 30801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:09:13.902855 30801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "9e813ea75d5e4c02af0b74f945c520d9"
format_stamp: "Formatted at 2026-09-28 23:09:13 on dist-test-slave-x2t8"
server_key: "8d053216346509b6f3b901963aaf377b"
server_key_iv: "7c92fb7427128d4251da11e8a40d6f82"
server_key_version: "encryptionkey@0"
I20260928 23:09:13.903172 30801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:13.918311 30801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:13.919955 30801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:13.920109 30801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:13.920320 30801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:13.920670 30801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:13.920725 30801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.920768 30801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:13.920794 30801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:13.926401 30801 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41545
I20260928 23:09:13.926443 30929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41545 every 8 connection(s)
I20260928 23:09:13.926757 30801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:09:13.931998 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30801
Sep 28 23:09:13 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636953, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:13.935648 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.927400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:34441 (local address 127.15.212.126:36273)
0928 23:09:13.927468 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.927470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.927665 (+ 195us) server_negotiation.cc:407] Connection header received
0928 23:09:13.928106 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.928107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.928117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.928167 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.929107 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.929585 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.930495 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.930678 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.930727 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.932845 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.932846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.932846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.932853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.934220 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.934699 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.934700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.934701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.934760 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.935003 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.935004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.935005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.935113 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.935190 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.935456 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.935531 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:09:13.935847 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.927401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:34441)
0928 23:09:13.927562 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.927593 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.928023 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.928243 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.928246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.928855 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:13.929024 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.929029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.929662 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.929663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.930422 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.930424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.930749 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.930749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.930842 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.931265 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:13.931273 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:13.932720 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:13.934321 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.934327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.934329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.934609 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.934833 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:13.934834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:13.934834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:13.934939 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:13.935181 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:13.935187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:13.935317 (+ 130us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.935422 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.935590 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260928 23:09:13.936262 30930 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36273
I20260928 23:09:13.936338 30930 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:13.936545 30930 heartbeater.cc:511] Master 127.15.212.126:36273 requested a full tablet report, sending...
I20260928 23:09:13.936941 30472 ts_manager.cc:194] Registered new tserver with Master: 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545)
I20260928 23:09:13.937436 30472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:34441
I20260928 23:09:13.938889 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.932395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.15.212.126:36273)
0928 23:09:13.932524 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.932525 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.932534 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:09:13.932607 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.932607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.932627 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.932704 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:13.933282 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.933761 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.934492 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.934711 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.934783 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.936160 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:13.936161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:13.936161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:13.936167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:13.937793 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.938137 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.938138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.938139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.938191 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:13.938396 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:13.938397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:13.938398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:13.938506 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:13.938674 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.938711 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.938789 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260928 23:09:13.939599 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:13.942423 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.939928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.15.212.126:36273)
0928 23:09:13.939974 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.939976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.940042 (+ 66us) server_negotiation.cc:407] Connection header received
0928 23:09:13.940148 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.940148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.940172 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.940226 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:13.940314 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.941047 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.941819 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.941993 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.942043 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.942173 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:13.942250 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:13.942316 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.942367 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.942383 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 23:09:13.942648 30472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:09:13.944176 30472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:09:13.946206 30578 tablet_service.cc:1467] Processing CreateTablet for tablet 895b2a7f3228498e9b5d6598796c8279 (DEFAULT_TABLE table=test-table [id=3d45351d1920448d8718bf6094d00c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.946409 30578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895b2a7f3228498e9b5d6598796c8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.947671 30552 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap starting.
I20260928 23:09:13.948002 30552 tablet_bootstrap.cc:654] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.948580 30552 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: No bootstrap required, opened a new log
I20260928 23:09:13.948647 30552 ts_tablet_manager.cc:1397] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:13.948947 30552 raft_consensus.cc:348] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.948997 30552 raft_consensus.cc:374] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.949023 30552 raft_consensus.cc:729] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24312d87b6d42fc984a66a8bd68ef06, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.949084 30552 consensus_queue.cc:260] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.949187 30552 ts_tablet_manager.cc:1428] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:13.950350 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.945949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35733 (local address 127.0.0.1:36608)
0928 23:09:13.946081 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.946090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.946141 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.946591 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.946592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.946594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.946633 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.946634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.947991 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.947992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.949171 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.949174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.950067 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.950067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.950180 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.950184 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.950236 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.950267 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260928 23:09:13.950407 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.946184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.15.212.66:35733)
0928 23:09:13.946349 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.946353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.946364 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:09:13.946406 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.946408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.946422 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.946504 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.946717 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.947813 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.949269 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.949961 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.950039 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.950094 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.950273 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.950315 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260928 23:09:13.950595 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.946092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.15.212.67:41545)
0928 23:09:13.946362 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:13.946366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:13.946375 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:09:13.946405 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:13.946407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:13.946422 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:13.946534 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:13.946682 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.947760 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.949268 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.949961 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.950395 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.950458 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:13.950485 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 23:09:13.950523 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":72,"threads_started":1}
I20260928 23:09:13.950428 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:13.945894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41545 (local address 127.0.0.1:56796)
0928 23:09:13.946033 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:13.946044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:13.946093 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:13.946562 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:13.946563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:13.946565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:13.946605 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.946606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.947885 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.947887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.949171 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:13.949174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:13.950073 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:13.950074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:13.950144 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:13.950161 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 23:09:13.950197 (+ 36us) client_negotiation.cc:240] Negotiation successful
0928 23:09:13.950272 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":77}
I20260928 23:09:13.951331 30732 tablet_service.cc:1467] Processing CreateTablet for tablet 895b2a7f3228498e9b5d6598796c8279 (DEFAULT_TABLE table=test-table [id=3d45351d1920448d8718bf6094d00c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.951427 30864 tablet_service.cc:1467] Processing CreateTablet for tablet 895b2a7f3228498e9b5d6598796c8279 (DEFAULT_TABLE table=test-table [id=3d45351d1920448d8718bf6094d00c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:13.951675 30864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895b2a7f3228498e9b5d6598796c8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.951675 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895b2a7f3228498e9b5d6598796c8279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:13.955185 30941 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Bootstrap starting.
I20260928 23:09:13.955713 30941 tablet_bootstrap.cc:654] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.956133 30941 log.cc:824] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.956363 30940 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Bootstrap starting.
I20260928 23:09:13.956959 30941 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: No bootstrap required, opened a new log
I20260928 23:09:13.957024 30940 tablet_bootstrap.cc:654] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:13.957038 30941 ts_tablet_manager.cc:1397] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:09:13.957592 30940 log.cc:824] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:13.958326 30940 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: No bootstrap required, opened a new log
I20260928 23:09:13.958396 30940 ts_tablet_manager.cc:1397] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 23:09:13.959228 30941 raft_consensus.cc:348] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.959386 30941 raft_consensus.cc:374] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.959421 30941 raft_consensus.cc:729] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a4a2cf1059046a3bd0e3d2825cc6aa2, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.959537 30941 consensus_queue.cc:260] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.959843 30941 ts_tablet_manager.cc:1428] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:09:13.959978 30798 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
I20260928 23:09:13.960244 30940 raft_consensus.cc:348] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.960402 30940 raft_consensus.cc:374] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:13.960462 30940 raft_consensus.cc:729] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e813ea75d5e4c02af0b74f945c520d9, State: Initialized, Role: FOLLOWER
I20260928 23:09:13.960578 30940 consensus_queue.cc:260] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:13.960840 30940 ts_tablet_manager.cc:1428] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 23:09:13.960861 30930 heartbeater.cc:503] Master 127.15.212.126:36273 was elected leader, sending a full tablet report...
W20260928 23:09:14.054849 30799 tablet.cc:2369] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:14.075788 30665 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:14.075925 30665 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:14.076354 30665 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
I20260928 23:09:14.079200 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.076642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:60603 (local address 127.15.212.66:35733)
0928 23:09:14.076717 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:14.076719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:14.076729 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:09:14.076803 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:14.076804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:14.076811 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:14.076876 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:14.077039 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.077528 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.078459 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.078828 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.078916 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:14.078924 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:14.079092 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 23:09:14.079140 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 23:09:14.079200 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.076569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35733 (local address 127.15.212.65:60603)
0928 23:09:14.076640 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.076652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:14.076707 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:14.076879 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:14.076880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:14.076882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:14.076963 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:14.076964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.077622 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.077624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:14.078354 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:14.078356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.078913 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.078914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:14.078996 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:14.079012 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 23:09:14.079084 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:09:14.079126 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 23:09:14.079352 30946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.076579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41545 (local address 127.15.212.65:48211)
0928 23:09:14.076771 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.076789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:14.076867 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:14.077022 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:14.077023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:14.077024 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:14.077114 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:14.077115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.077745 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.077746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:14.078483 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:14.078484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.079071 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.079072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:14.079141 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:14.079146 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:09:14.079215 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 23:09:14.079252 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":12,"thread_start_us":112,"threads_started":1}
I20260928 23:09:14.079341 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.076656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48211 (local address 127.15.212.67:41545)
0928 23:09:14.076708 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:14.076711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:14.076856 (+ 145us) server_negotiation.cc:407] Connection header received
0928 23:09:14.076940 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:14.076942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:14.076957 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:14.077018 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:14.077186 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.077660 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.078597 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.078973 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.079067 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:14.079076 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:14.079233 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 23:09:14.079274 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:09:14.079555 30752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895b2a7f3228498e9b5d6598796c8279" candidate_uuid: "d24312d87b6d42fc984a66a8bd68ef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" is_pre_election: true
I20260928 23:09:14.079708 30752 raft_consensus.cc:2454] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d24312d87b6d42fc984a66a8bd68ef06 in term 0.
I20260928 23:09:14.079770 30884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895b2a7f3228498e9b5d6598796c8279" candidate_uuid: "d24312d87b6d42fc984a66a8bd68ef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e813ea75d5e4c02af0b74f945c520d9" is_pre_election: true
I20260928 23:09:14.079913 30549 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, d24312d87b6d42fc984a66a8bd68ef06; no voters:
I20260928 23:09:14.079944 30884 raft_consensus.cc:2454] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d24312d87b6d42fc984a66a8bd68ef06 in term 0.
I20260928 23:09:14.080058 30665 raft_consensus.cc:2781] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:14.080152 30665 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:14.080190 30665 raft_consensus.cc:3037] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:14.080960 30665 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:14.081110 30665 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 1 election: Requested vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
I20260928 23:09:14.081238 30884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895b2a7f3228498e9b5d6598796c8279" candidate_uuid: "d24312d87b6d42fc984a66a8bd68ef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e813ea75d5e4c02af0b74f945c520d9"
I20260928 23:09:14.081281 30752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895b2a7f3228498e9b5d6598796c8279" candidate_uuid: "d24312d87b6d42fc984a66a8bd68ef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
I20260928 23:09:14.081312 30884 raft_consensus.cc:3037] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:14.081368 30752 raft_consensus.cc:3037] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:14.082058 30752 raft_consensus.cc:2454] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24312d87b6d42fc984a66a8bd68ef06 in term 1.
I20260928 23:09:14.082073 30884 raft_consensus.cc:2454] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24312d87b6d42fc984a66a8bd68ef06 in term 1.
I20260928 23:09:14.082226 30549 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, d24312d87b6d42fc984a66a8bd68ef06; no voters:
I20260928 23:09:14.082338 30665 raft_consensus.cc:2781] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:14.082491 30665 raft_consensus.cc:686] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 LEADER]: Becoming Leader. State: Replica: d24312d87b6d42fc984a66a8bd68ef06, State: Running, Role: LEADER
I20260928 23:09:14.082576 30665 consensus_queue.cc:237] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:14.083310 30472 catalog_manager.cc:5495] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 reported cstate change: term changed from 0 to 1, leader changed from <none> to d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65). New cstate: current_term: 1 leader_uuid: "d24312d87b6d42fc984a66a8bd68ef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } health_report { overall_health: HEALTHY } } }
I20260928 23:09:14.094789 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.092745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49482 (local address 127.15.212.65:42213)
0928 23:09:14.092783 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:14.092785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:14.092791 (+ 6us) server_negotiation.cc:407] Connection header received
0928 23:09:14.092875 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:14.092876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:14.092889 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:14.092934 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:14.093022 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.093485 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.094146 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:14.094328 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:14.094375 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:14.094497 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:14.094583 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:14.094633 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:14.094675 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 23:09:14.094691 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:09:14.096210 30752 raft_consensus.cc:1261] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Refusing update from remote peer d24312d87b6d42fc984a66a8bd68ef06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:14.096210 30884 raft_consensus.cc:1261] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Refusing update from remote peer d24312d87b6d42fc984a66a8bd68ef06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:14.096570 30665 consensus_queue.cc:1035] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:14.096699 30947 consensus_queue.cc:1035] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:14.098865 30950 mvcc.cc:205] Tried to move back new op lower bound from 7334448963975659520 to 7334448963922673664. Current Snapshot: MvccSnapshot[applied={T|T < 7334448963975659520}]
I20260928 23:09:14.099069 30951 mvcc.cc:205] Tried to move back new op lower bound from 7334448963975659520 to 7334448963922673664. Current Snapshot: MvccSnapshot[applied={T|T < 7334448963975659520}]
I20260928 23:09:14.101269 30953 mvcc.cc:205] Tried to move back new op lower bound from 7334448963975659520 to 7334448963922673664. Current Snapshot: MvccSnapshot[applied={T|T < 7334448963975659520 or (T in {7334448963975659520})}]
I20260928 23:09:14.101503 30471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:14.101590 30471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:14.102294 30471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Sending DeleteTablet for 3 replicas of tablet 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:14.102572 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30510
I20260928 23:09:14.102684 30864 tablet_service.cc:1514] Processing DeleteTablet for tablet 895b2a7f3228498e9b5d6598796c8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:14 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:56796
I20260928 23:09:14.103080 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet 895b2a7f3228498e9b5d6598796c8279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:14 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:36608
I20260928 23:09:14.103279 30961 tablet_replica.cc:330] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: stopping tablet replica
I20260928 23:09:14.103351 30961 raft_consensus.cc:2229] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:14.103415 30961 raft_consensus.cc:2258] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:14.103772 30961 ts_tablet_manager.cc:1905] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:14.105400 30961 ts_tablet_manager.cc:1918] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:14.105474 30961 log.cc:1196] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:14.105818 30961 ts_tablet_manager.cc:1939] T 895b2a7f3228498e9b5d6598796c8279 P 3a4a2cf1059046a3bd0e3d2825cc6aa2: Deleting consensus metadata
I20260928 23:09:14.106348 30459 catalog_manager.cc:4841] TS 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): tablet 895b2a7f3228498e9b5d6598796c8279 (table test-table [id=3d45351d1920448d8718bf6094d00c67]) successfully deleted
W20260928 23:09:14.108737 30456 connection.cc:535] client connection to 127.15.212.65:42213 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 23:09:14.108824 30456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:09:14.109261 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.109151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:42213 (local address 127.0.0.1:49488)
0928 23:09:14.109203 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.109235 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:09:14.109269 30960 tablet_replica.cc:330] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: stopping tablet replica
I20260928 23:09:14.109382 30960 raft_consensus.cc:2229] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:14.109485 30960 raft_consensus.cc:2258] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:14.109664 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30666
W20260928 23:09:14.109396 30456 catalog_manager.cc:4584] TS d24312d87b6d42fc984a66a8bd68ef06 (127.15.212.65:42213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 895b2a7f3228498e9b5d6598796c8279: Network error: Client connection negotiation failed: client connection to 127.15.212.65:42213: connect: Connection refused (error 111)
I20260928 23:09:14.110983 30960 ts_tablet_manager.cc:1905] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:14.112919 30960 ts_tablet_manager.cc:1918] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:14.112998 30960 log.cc:1196] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:14.113276 30960 ts_tablet_manager.cc:1939] T 895b2a7f3228498e9b5d6598796c8279 P 9e813ea75d5e4c02af0b74f945c520d9: Deleting consensus metadata
I20260928 23:09:14.113790 30456 catalog_manager.cc:4841] TS 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): tablet 895b2a7f3228498e9b5d6598796c8279 (table test-table [id=3d45351d1920448d8718bf6094d00c67]) successfully deleted
I20260928 23:09:14.115497 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30801
I20260928 23:09:14.119742 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 30439
I20260928 23:09:14.126207 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--webserver_interface=127.15.212.126
--webserver_port=45749
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:36273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:14.206604 30963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:14.206787 30963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:14.206808 30963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:14.207888 30963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:14.207926 30963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:14.207940 30963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:14.207953 30963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:14.208003 30963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:14.209483 30963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:36273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=45749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:14.209723 30963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:14.209944 30963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:14.214622 30969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.214655 30963 server_base.cc:1034] running on GCE node
W20260928 23:09:14.214596 30971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.214608 30968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.214967 30963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:14.215253 30963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:14.216485 30963 hybrid_clock.cc:648] HybridClock initialized: now 1790636954216462 us; error 34 us; skew 500 ppm
Sep 28 23:09:14 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:14.218748 30963 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:14.219167 30963 webserver.cc:460] Webserver started at http://127.15.212.126:45749/ using document root <none> and password file <none>
I20260928 23:09:14.219350 30963 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:14.219388 30963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:14.220610 30963 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:14.221218 30978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.221393 30963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:14.221474 30963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "00ea5f9ff60c4a42ac1425b697ef5431"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "b2f323103481f51188ede7be9db79143"
server_key_iv: "0abb802d36dafced75f7c56158373818"
server_key_version: "encryptionkey@0"
I20260928 23:09:14.221757 30963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:14.233443 30963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:14.235669 30963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:14.235770 30963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:14.239466 30963 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:36273
I20260928 23:09:14.239513 31030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:36273 every 8 connection(s)
I20260928 23:09:14.239847 30963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:14.241364 31031 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:09:14.241518 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 30963
I20260928 23:09:14.241780 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:42213
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=45439
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:14.242249 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap starting.
I20260928 23:09:14.250650 31031 log.cc:824] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:14.252861 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:14.253145 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431: Bootstrap complete.
I20260928 23:09:14.255245 31031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:14.255476 31031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Initialized, Role: FOLLOWER
I20260928 23:09:14.255673 31031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:14.255774 31031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:14.255826 31031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:14.255873 31031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 2 FOLLOWER]: Advancing to term 3
I20260928 23:09:14.256883 31031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:14.256999 31031 leader_election.cc:302] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ea5f9ff60c4a42ac1425b697ef5431; no voters:
I20260928 23:09:14.257202 31031 leader_election.cc:288] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 23:09:14.257236 31035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 23:09:14.257341 31035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [term 3 LEADER]: Becoming Leader. State: Replica: 00ea5f9ff60c4a42ac1425b697ef5431, State: Running, Role: LEADER
I20260928 23:09:14.257477 31031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:14.257506 31035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } }
I20260928 23:09:14.257879 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:14.257974 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:14.257877 31035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ea5f9ff60c4a42ac1425b697ef5431. Latest consensus state: current_term: 3 leader_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ea5f9ff60c4a42ac1425b697ef5431" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36273 } } }
I20260928 23:09:14.258167 31035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ea5f9ff60c4a42ac1425b697ef5431 [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:14.258762 31042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:14.259578 31042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:14.259696 31042 catalog_manager.cc:1259] Loaded cluster ID: 807f73317a994990ac5bf38f03d3fb40
I20260928 23:09:14.259738 31042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 23:09:14.260200 31042 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f56fd86c9ad google::LogMessage::Fail() at ??:0
@ 0x7f56fd86f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f56fd86c4de google::LogMessage::Flush() at ??:0
@ 0x7f56fd86da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f570578d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f57057968b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f56ff73a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f56ff72f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f57005936db start_thread at ??:0
@ 0x7f56fc36a61f clone at ??:0
W20260928 23:09:14.335608 31033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:14.335789 31033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:14.335815 31033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:14.337056 31033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:14.337096 31033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:14.337124 31033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:14.338531 31033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:42213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=45439
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:14.338749 31033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:14.338992 31033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:14.343860 31067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.343887 31068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.343991 31070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.344213 31033 server_base.cc:1034] running on GCE node
I20260928 23:09:14.344426 31033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:14.344733 31033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:14.345883 31033 hybrid_clock.cc:648] HybridClock initialized: now 1790636954345867 us; error 33 us; skew 500 ppm
Sep 28 23:09:14 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:14.348371 31033 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:14.348847 31033 webserver.cc:460] Webserver started at http://127.15.212.65:45439/ using document root <none> and password file <none>
I20260928 23:09:14.349066 31033 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:14.349141 31033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:14.350456 31033 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:14.351147 31077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.351325 31033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:14.351418 31033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d24312d87b6d42fc984a66a8bd68ef06"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "2699ab7ab40443ba5ab3b71581ed31b5"
server_key_iv: "ac3a9b697036434ce8f8319674640697"
server_key_version: "encryptionkey@0"
I20260928 23:09:14.351742 31033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:14.363435 31033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:14.365646 31033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:14.365797 31033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:14.366007 31033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:14.366385 31084 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 23:09:14.368101 31033 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 23:09:14.368165 31033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 23:09:14.368206 31033 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 23:09:14.368872 31033 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 23:09:14.368927 31033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:14.368991 31084 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap starting.
I20260928 23:09:14.376355 31033 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:42213
I20260928 23:09:14.376536 31191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:42213 every 8 connection(s)
I20260928 23:09:14.376813 31033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:14.378898 31084 log.cc:824] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:14.381014 31084 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:14.381286 31084 tablet_bootstrap.cc:492] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Bootstrap complete.
I20260928 23:09:14.382181 31084 ts_tablet_manager.cc:1397] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.010s
I20260928 23:09:14.383131 31084 raft_consensus.cc:348] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:14.383345 31084 raft_consensus.cc:729] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24312d87b6d42fc984a66a8bd68ef06, State: Initialized, Role: FOLLOWER
I20260928 23:09:14.383467 31084 consensus_queue.cc:260] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:14.383723 31084 ts_tablet_manager.cc:1428] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20260928 23:09:14.386962 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31033
I20260928 23:09:14.387243 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:35733
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=34139
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:14.467382 31197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:14.467603 31197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:14.467639 31197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:14.468827 31197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:14.468887 31197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:14.468931 31197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:14.471336 31197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:35733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=34139
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:14.471591 31197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:14.471807 31197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:14.474073 31194 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:09:14.377637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:36633)
0928 23:09:14.377881 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.377935 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:14.378624 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:14.473884 (+ 95260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":110,"threads_started":1}
W20260928 23:09:14.474321 31192 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:36273 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:36273: Client connection negotiation failed: client connection to 127.15.212.126:36273: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:09:14.474670 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.474558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:54869)
0928 23:09:14.474610 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.474650 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:09:14.475010 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.474923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:39785)
0928 23:09:14.474969 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.474994 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 23:09:14.475124 31192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:09:14.477279 31203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.477322 31205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.477341 31202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.477432 31197 server_base.cc:1034] running on GCE node
I20260928 23:09:14.477682 31197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:14.477867 31197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:14.479002 31197 hybrid_clock.cc:648] HybridClock initialized: now 1790636954478982 us; error 32 us; skew 500 ppm
Sep 28 23:09:14 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:14.481272 31197 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:14.481633 31197 webserver.cc:460] Webserver started at http://127.15.212.66:34139/ using document root <none> and password file <none>
I20260928 23:09:14.481817 31197 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:14.481856 31197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:14.483031 31197 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:14.483531 31212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.483736 31197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:09:14.483830 31197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2"
format_stamp: "Formatted at 2026-09-28 23:09:12 on dist-test-slave-x2t8"
server_key: "34dd5543913947ef9e350b62fddc84b7"
server_key_iv: "541254a90ba18192881e2a85e9e4c248"
server_key_version: "encryptionkey@0"
I20260928 23:09:14.484156 31197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:14.508019 31197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:14.509872 31197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:14.510033 31197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:14.510251 31197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:14.510541 31197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:14.510574 31197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.510617 31197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:14.510643 31197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.516680 31197 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35733
I20260928 23:09:14.516780 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35733 every 8 connection(s)
I20260928 23:09:14.517104 31197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:14.518100 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.517753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:48795)
0928 23:09:14.517891 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.517961 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
W20260928 23:09:14.518486 31326 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:36273 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:36273: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
I20260928 23:09:14.518774 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.518681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:52793)
0928 23:09:14.518731 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.518757 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 23:09:14.519012 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.518964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:37803)
0928 23:09:14.518984 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.519000 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260928 23:09:14.519125 31326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 23:09:14.522341 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31197
I20260928 23:09:14.522598 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:41545
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:36273
--webserver_port=44207
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:14.602527 31329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:14.602722 31329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:14.602747 31329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:14.603894 31329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:14.603937 31329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:14.604005 31329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:14.605525 31329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:41545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=44207
--tserver_master_addrs=127.15.212.126:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:14.605777 31329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:14.606014 31329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:14.610935 31335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:14.610953 31334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.610997 31329 server_base.cc:1034] running on GCE node
W20260928 23:09:14.610954 31337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:14.611380 31329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:14.611614 31329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:14.612807 31329 hybrid_clock.cc:648] HybridClock initialized: now 1790636954612788 us; error 31 us; skew 500 ppm
Sep 28 23:09:14 dist-test-slave-x2t8 krb5kdc[29895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:14.615283 31329 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:14.615697 31329 webserver.cc:460] Webserver started at http://127.15.212.67:44207/ using document root <none> and password file <none>
I20260928 23:09:14.615936 31329 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:14.615983 31329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:14.617424 31329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:14.617990 31344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.618168 31329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:14.618269 31329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "9e813ea75d5e4c02af0b74f945c520d9"
format_stamp: "Formatted at 2026-09-28 23:09:13 on dist-test-slave-x2t8"
server_key: "8d053216346509b6f3b901963aaf377b"
server_key_iv: "7c92fb7427128d4251da11e8a40d6f82"
server_key_version: "encryptionkey@0"
I20260928 23:09:14.618631 31329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:14.648834 31329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:14.650556 31329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:14.650705 31329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:14.650918 31329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:14.651232 31329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:14.651285 31329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.651327 31329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:14.651355 31329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:14.657101 31329 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41545
I20260928 23:09:14.657186 31457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41545 every 8 connection(s)
I20260928 23:09:14.657516 31329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:09:14.658479 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.658096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:56519)
0928 23:09:14.658234 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.658338 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260928 23:09:14.658845 31458 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:36273 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:36273: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
I20260928 23:09:14.659088 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.659008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:48829)
0928 23:09:14.659047 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.659072 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260928 23:09:14.659325 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:14.659255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:33257)
0928 23:09:14.659293 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:14.659313 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 23:09:14.659435 31458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 23:09:14.667438 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31329
I20260928 23:09:15.475661 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.475451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:43475)
0928 23:09:15.475570 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:15.475600 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260928 23:09:15.519672 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.519465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:57249)
0928 23:09:15.519588 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:15.519621 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260928 23:09:15.660012 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.659760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:59437)
0928 23:09:15.659895 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:15.659948 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260928 23:09:15.728951 31465 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:15.729103 31465 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:15.729525 31465 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
Sep 28 23:09:15 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.66@KRBTEST.COM
Sep 28 23:09:15 dist-test-slave-x2t8 krb5kdc[29895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.67@KRBTEST.COM
I20260928 23:09:15.740489 31466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.729677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35733 (local address 127.15.212.65:32975)
0928 23:09:15.729800 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:15.729811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:15.729875 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:15.731543 (+ 1668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:15.731544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:15.731923 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:15.732101 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:15.732107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.733276 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.733277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:15.734073 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:15.734076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.734657 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:15.734791 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:15.735355 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:15.735365 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:15.737007 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:15.738908 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:15.738914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:15.738916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:15.739136 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:15.739408 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:15.739415 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:15.739415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:15.739550 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:15.740004 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:15.740012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:15.740168 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 23:09:15.740260 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 23:09:15.740370 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":32}
I20260928 23:09:15.740624 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.729769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:32975 (local address 127.15.212.66:35733)
0928 23:09:15.730802 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:15.730814 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:15.730847 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:09:15.731359 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:15.731362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:15.731401 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:15.731567 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:15.732230 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.733150 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734178 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.734559 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734673 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:15.737124 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:15.737129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:15.737129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:15.737141 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:15.738793 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:15.739221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:15.739224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:15.739224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:15.739320 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:15.739609 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:15.739610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:15.739611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:15.739905 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:15.740012 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:15.740339 (+ 327us) server_negotiation.cc:299] Negotiation successful
0928 23:09:15.740502 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":58,"threads_started":1}
I20260928 23:09:15.740899 31462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.729677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41545 (local address 127.15.212.65:50403)
0928 23:09:15.730581 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:15.730592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:15.730673 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:15.731540 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:15.731542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:15.731921 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:15.732101 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:15.732107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.733291 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.733292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:15.734075 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:15.734077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.734655 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:15.734784 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:15.735519 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:15.735524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:15.737373 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:15.739557 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:15.739558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:15.739558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:15.739737 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:15.740004 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:15.740005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:15.740006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:15.740069 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:15.740579 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:15.740580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:15.740624 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:09:15.740705 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 23:09:15.740809 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":54,"thread_start_us":69,"threads_started":1}
I20260928 23:09:15.741019 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:15.729953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:50403 (local address 127.15.212.67:41545)
0928 23:09:15.730118 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:15.730149 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:15.730663 (+ 514us) server_negotiation.cc:407] Connection header received
0928 23:09:15.731309 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:15.731312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:15.731355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:15.731516 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:15.732221 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.733156 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734177 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:15.734562 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:15.734686 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:15.737718 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:15.737721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:15.737721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:15.737730 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:15.739479 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:15.739817 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:15.739820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:15.739821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:15.739926 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:15.740138 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:15.740138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:15.740139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:15.740424 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:15.740532 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:15.740788 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 23:09:15.740918 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":52,"threads_started":1}
W20260928 23:09:15.741572 31081 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
W20260928 23:09:15.742095 31078 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:15.742138 31078 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d24312d87b6d42fc984a66a8bd68ef06; no voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9
I20260928 23:09:15.742259 31465 raft_consensus.cc:2726] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:09:16.476318 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:16.476114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:34383)
0928 23:09:16.476241 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:16.476273 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260928 23:09:16.520319 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:16.520113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:45141)
0928 23:09:16.520249 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:16.520283 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260928 23:09:16.660647 31471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:16.660418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:44877)
0928 23:09:16.660585 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:16.660614 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260928 23:09:17.477048 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:17.476839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:56155)
0928 23:09:17.476972 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:17.477003 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260928 23:09:17.521005 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:17.520809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:52901)
0928 23:09:17.520933 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:17.520968 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260928 23:09:17.661286 31474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:17.661081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:54859)
0928 23:09:17.661211 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:17.661247 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260928 23:09:17.838502 31475 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:17.838599 31475 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:17.838827 31475 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
W20260928 23:09:17.839215 31078 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
W20260928 23:09:17.839294 31081 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:17.839382 31081 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d24312d87b6d42fc984a66a8bd68ef06; no voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9
I20260928 23:09:17.839542 31475 raft_consensus.cc:2726] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:09:18.477689 31476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:18.477496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:36117)
0928 23:09:18.477607 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:18.477641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260928 23:09:18.521682 31477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:18.521464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:34975)
0928 23:09:18.521607 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:18.521641 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260928 23:09:18.661924 31478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:18.661733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:51947)
0928 23:09:18.661845 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:18.661882 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260928 23:09:19.478458 31479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:19.478215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:48245)
0928 23:09:19.478362 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:19.478402 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260928 23:09:19.522344 31480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:19.522132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:59569)
0928 23:09:19.522260 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:19.522300 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260928 23:09:19.662606 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:19.662371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:37989)
0928 23:09:19.662528 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:19.662564 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260928 23:09:19.998052 31482 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:19.998179 31482 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:19.998392 31482 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
W20260928 23:09:19.998737 31078 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
W20260928 23:09:19.998822 31081 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:19.998852 31078 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d24312d87b6d42fc984a66a8bd68ef06; no voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9
I20260928 23:09:19.998926 31482 raft_consensus.cc:2726] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:09:20.479290 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:20.479034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:34373)
0928 23:09:20.479205 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:20.479240 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260928 23:09:20.523023 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:20.522806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:34023)
0928 23:09:20.522945 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:20.522982 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260928 23:09:20.663308 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:20.663076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:57965)
0928 23:09:20.663213 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:20.663255 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260928 23:09:21.480302 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:21.479918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:39247)
0928 23:09:21.480193 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:21.480238 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":87,"threads_started":1}
I20260928 23:09:21.523747 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:21.523513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:52273)
0928 23:09:21.523662 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:21.523703 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260928 23:09:21.664002 31488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:21.663793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:53215)
0928 23:09:21.663912 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:21.663950 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260928 23:09:22.481142 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:22.480876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:41847)
0928 23:09:22.481051 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:22.481095 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260928 23:09:22.524513 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:22.524230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:55297)
0928 23:09:22.524363 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:22.524401 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260928 23:09:22.664798 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:22.664595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:51947)
0928 23:09:22.664716 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:22.664757 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260928 23:09:23.481798 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:23.481597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:36925)
0928 23:09:23.481723 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:23.481752 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260928 23:09:23.525089 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:23.524901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:60513)
0928 23:09:23.525021 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:23.525051 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260928 23:09:23.665354 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:23.665181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:45907)
0928 23:09:23.665293 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:23.665320 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260928 23:09:24.482556 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:24.482265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:52695)
0928 23:09:24.482470 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:24.482509 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260928 23:09:24.525772 31496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:24.525560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:60959)
0928 23:09:24.525687 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:24.525726 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260928 23:09:24.665993 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:24.665768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:50275)
0928 23:09:24.665919 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:24.665956 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":55,"threads_started":1}
I20260928 23:09:24.785880 31498 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:24.786007 31498 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:24.786202 31498 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
W20260928 23:09:24.786588 31078 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
W20260928 23:09:24.786664 31081 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:24.786711 31081 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d24312d87b6d42fc984a66a8bd68ef06; no voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9
I20260928 23:09:24.786804 31498 raft_consensus.cc:2726] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:09:25.483186 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:25.482995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:42551)
0928 23:09:25.483107 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:25.483139 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260928 23:09:25.526405 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:25.526208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:38669)
0928 23:09:25.526330 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:25.526363 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260928 23:09:25.666671 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:25.666429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:36663)
0928 23:09:25.666571 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:25.666627 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260928 23:09:26.483855 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:26.483646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:46265)
0928 23:09:26.483766 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:26.483807 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260928 23:09:26.527020 31503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:26.526832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:44197)
0928 23:09:26.526947 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:26.526979 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260928 23:09:26.667279 31504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:26.667084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:41465)
0928 23:09:26.667208 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:26.667240 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260928 23:09:27.484642 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:27.484338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:53899)
0928 23:09:27.484556 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:27.484593 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":60,"threads_started":1}
I20260928 23:09:27.518822 31506 raft_consensus.cc:482] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:27.518908 31506 raft_consensus.cc:504] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e813ea75d5e4c02af0b74f945c520d9" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41545 } } peers { permanent_uuid: "3a4a2cf1059046a3bd0e3d2825cc6aa2" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35733 } } peers { permanent_uuid: "d24312d87b6d42fc984a66a8bd68ef06" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 42213 } }
I20260928 23:09:27.519088 31506 leader_election.cc:288] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545), 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733)
W20260928 23:09:27.519527 31081 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a4a2cf1059046a3bd0e3d2825cc6aa2 (127.15.212.66:35733): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
W20260928 23:09:27.519608 31078 leader_election.cc:341] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e813ea75d5e4c02af0b74f945c520d9 (127.15.212.67:41545): Not found: Tablet not found: 895b2a7f3228498e9b5d6598796c8279
I20260928 23:09:27.519709 31078 leader_election.cc:302] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d24312d87b6d42fc984a66a8bd68ef06; no voters: 3a4a2cf1059046a3bd0e3d2825cc6aa2, 9e813ea75d5e4c02af0b74f945c520d9
I20260928 23:09:27.519789 31506 raft_consensus.cc:2726] T 895b2a7f3228498e9b5d6598796c8279 P d24312d87b6d42fc984a66a8bd68ef06 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:09:27.527697 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:27.527469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:33857)
0928 23:09:27.527621 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:27.527658 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260928 23:09:27.667892 31508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:27.667690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:33883)
0928 23:09:27.667816 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:27.667852 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
I20260928 23:09:28.485359 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:28.485129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:34321)
0928 23:09:28.485265 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:28.485310 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260928 23:09:28.528582 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:28.528300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:52065)
0928 23:09:28.528478 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:28.528523 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260928 23:09:28.668641 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:28.668386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:35305)
0928 23:09:28.668558 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:28.668596 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":118,"threads_started":1}
I20260928 23:09:29.486274 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:29.486046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.65:48527)
0928 23:09:29.486188 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:29.486228 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260928 23:09:29.529543 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:29.529303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.66:60971)
0928 23:09:29.529452 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:29.529498 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260928 23:09:29.669461 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:29.669240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36273 (local address 127.15.212.67:34509)
0928 23:09:29.669379 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:29.669420 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:36273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260928 23:09:29.678997 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31033
I20260928 23:09:29.683682 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31197
I20260928 23:09:29.687980 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31329
2026-09-28T23:09:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18561 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:09:29 dist-test-slave-x2t8 krb5kdc[31517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:09:29 dist-test-slave-x2t8 krb5kdc[31517](info): set up 2 sockets
Sep 28 23:09:29 dist-test-slave-x2t8 krb5kdc[31517](info): commencing operation
krb5kdc: starting...
W20260928 23:09:30.940670 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:09:31.060735 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:32799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:31.139385 31530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:31.139637 31530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:31.139672 31530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:31.140782 31530 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 23:09:31.140853 31530 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 23:09:31.140874 31530 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 23:09:31.140892 31530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:31.140924 31530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:31.140952 31530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:31.140974 31530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:31.141000 31530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:31.142423 31530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:32799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:31.142726 31530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:31.142947 31530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:31.148300 31538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.148357 31530 server_base.cc:1034] running on GCE node
W20260928 23:09:31.148365 31535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:31.148365 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.148792 31530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:31.149089 31530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:31.150233 31530 hybrid_clock.cc:648] HybridClock initialized: now 1790636971150223 us; error 29 us; skew 500 ppm
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:31.152645 31530 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:31.153110 31530 webserver.cc:460] Webserver started at http://127.15.212.126:37457/ using document root <none> and password file <none>
I20260928 23:09:31.153357 31530 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:31.153425 31530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:31.153577 31530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:31.154597 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "6046e2d01d204bb59888504e00ea65ba"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "8a173a666488bdcd430a4fb22f419b56"
server_key_iv: "7db8b8b288bbe49d5a7c7c506f900e74"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.154953 31530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "6046e2d01d204bb59888504e00ea65ba"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "8a173a666488bdcd430a4fb22f419b56"
server_key_iv: "7db8b8b288bbe49d5a7c7c506f900e74"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.156242 31530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 23:09:31.157037 31545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.157213 31530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.157302 31530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "6046e2d01d204bb59888504e00ea65ba"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "8a173a666488bdcd430a4fb22f419b56"
server_key_iv: "7db8b8b288bbe49d5a7c7c506f900e74"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.157393 31530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:31.181967 31530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:31.183830 31530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:31.184007 31530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:31.188083 31530 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:32799
I20260928 23:09:31.188122 31597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:32799 every 8 connection(s)
I20260928 23:09:31.188521 31530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:31.189402 31598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:31.191603 31598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap starting.
I20260928 23:09:31.192130 31598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:31.192370 31598 log.cc:824] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:31.193105 31598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: No bootstrap required, opened a new log
I20260928 23:09:31.194636 31598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:31.194773 31598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:31.194795 31598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Initialized, Role: FOLLOWER
I20260928 23:09:31.194900 31598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:31.194985 31598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:31.195020 31598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:31.195073 31598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:31.195456 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31530
I20260928 23:09:31.195542 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:09:31.195669 31598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:31.195749 16209 external_mini_cluster.cc:1428] Setting key a03d104c4ea297e769206598056bb17c
I20260928 23:09:31.195792 31598 leader_election.cc:302] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6046e2d01d204bb59888504e00ea65ba; no voters:
I20260928 23:09:31.195971 31598 leader_election.cc:288] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:09:31.196031 31601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:31.196185 31601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 LEADER]: Becoming Leader. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Running, Role: LEADER
I20260928 23:09:31.196238 31598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:31.196274 31601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:31.196918 31601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6046e2d01d204bb59888504e00ea65ba. Latest consensus state: current_term: 1 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:09:31.197001 31601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:31.196928 31602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:09:31.197190 31602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
W20260928 23:09:31.198143 31618 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:09:31.198199 31618 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:09:31.198247 31612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:31.198406 31612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:31.200855 31612 catalog_manager.cc:1347] Generated new cluster ID: 90f66fa3b0eb4697aa59edfef23d3064
I20260928 23:09:31.200909 31612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:31.207454 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.196457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.15.212.126:32799)
0928 23:09:31.196807 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.196818 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.196859 (+ 41us) server_negotiation.cc:407] Connection header received
0928 23:09:31.196987 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.196995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.197057 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.197849 (+ 792us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:31.198208 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.199396 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.200572 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.201159 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.201284 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.203612 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:31.203617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:31.203617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:31.203628 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:31.205893 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.206290 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.206293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.206293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.206384 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.206617 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.206618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.206618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.206855 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:31.207005 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.207127 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.207272 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":76,"threads_started":1}
I20260928 23:09:31.208587 31612 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:09:31.209122 31612 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:31.217208 31612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Generated new TSK 0
I20260928 23:09:31.217439 31612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:09:31.247598 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:31.347389 31624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:31.347585 31624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:31.347609 31624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:31.348780 31624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:31.348819 31624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:31.348845 31624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:31.350267 31624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:31.350507 31624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:31.350751 31624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:31.355655 31629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:31.355656 31632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.355684 31624 server_base.cc:1034] running on GCE node
W20260928 23:09:31.355651 31630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.356153 31624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:31.356412 31624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:31.357607 31624 hybrid_clock.cc:648] HybridClock initialized: now 1790636971357598 us; error 30 us; skew 500 ppm
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:31.360204 31624 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:31.360703 31624 webserver.cc:460] Webserver started at http://127.15.212.65:36043/ using document root <none> and password file <none>
I20260928 23:09:31.360935 31624 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:31.361003 31624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:31.361158 31624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:31.362211 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d5dd92f1bf8c0d5959507e11e7edc334"
server_key_iv: "77296a8942ff3e442fb52c6422a0b43a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.362582 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d5dd92f1bf8c0d5959507e11e7edc334"
server_key_iv: "77296a8942ff3e442fb52c6422a0b43a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.363847 31624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:09:31.364595 31639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.364769 31624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:31.364867 31624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d5dd92f1bf8c0d5959507e11e7edc334"
server_key_iv: "77296a8942ff3e442fb52c6422a0b43a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.364948 31624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:31.379853 31624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:31.381969 31624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:31.382150 31624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:31.382406 31624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:31.382754 31624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:31.382810 31624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.382850 31624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:31.382879 31624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.388796 31624 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:44051
I20260928 23:09:31.388849 31752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:44051 every 8 connection(s)
I20260928 23:09:31.389183 31624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:31.393181 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31624
I20260928 23:09:31.393306 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:09:31.393533 16209 external_mini_cluster.cc:1428] Setting key fff7b8db95a62773737a543bcdc7e91e
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:31.399546 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.389892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48703 (local address 127.15.212.126:32799)
0928 23:09:31.389945 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.389947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.390154 (+ 207us) server_negotiation.cc:407] Connection header received
0928 23:09:31.390626 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.390627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.390641 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.390700 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:31.391637 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.392135 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.393302 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.393588 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.393671 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.396236 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:31.396238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:31.396238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:31.396246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:31.397932 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.398433 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.398434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.398434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.398487 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.398790 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.398792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.398792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.398925 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:31.399007 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.399361 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.399454 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:09:31.399771 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.389847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:48703)
0928 23:09:31.390005 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.390048 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.390542 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.390768 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.390772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.391335 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:31.391545 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.391550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.392232 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.392233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.393183 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.393186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.393703 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.393705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.393825 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.394565 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:31.394574 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:31.396107 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:31.398008 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.398016 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.398019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.398357 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.398557 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.398558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.398558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.398715 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.398999 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:31.399006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:31.399204 (+ 198us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.399338 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.399509 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260928 23:09:31.400290 31753 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:31.400395 31753 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:31.400606 31753 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:31.401182 31562 ts_manager.cc:194] Registered new tserver with Master: d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051)
I20260928 23:09:31.402174 31562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:48703
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:09:31.419885 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:31.501286 31758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:31.501492 31758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:31.501514 31758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:31.502652 31758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:31.502697 31758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:31.502722 31758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:31.504141 31758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:31.504379 31758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:31.504637 31758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:31.509783 31764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:31.509778 31763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.509831 31758 server_base.cc:1034] running on GCE node
W20260928 23:09:31.509783 31766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.510185 31758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:31.510464 31758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:31.511618 31758 hybrid_clock.cc:648] HybridClock initialized: now 1790636971511600 us; error 35 us; skew 500 ppm
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:31.514196 31758 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:31.514633 31758 webserver.cc:460] Webserver started at http://127.15.212.66:34735/ using document root <none> and password file <none>
I20260928 23:09:31.514817 31758 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:31.514856 31758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:31.514943 31758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:31.515931 31758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "6751b5727f54403da9d44a8cc4e52ca7"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d7c4bb53310b87d2fcaa565e806d9e4b"
server_key_iv: "c9d34db19278c1e6967dbd16db9cf78a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.516229 31758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "6751b5727f54403da9d44a8cc4e52ca7"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d7c4bb53310b87d2fcaa565e806d9e4b"
server_key_iv: "c9d34db19278c1e6967dbd16db9cf78a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.517573 31758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:09:31.518230 31773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.518395 31758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:31.518476 31758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "6751b5727f54403da9d44a8cc4e52ca7"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d7c4bb53310b87d2fcaa565e806d9e4b"
server_key_iv: "c9d34db19278c1e6967dbd16db9cf78a"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.518532 31758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:31.537536 31758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:31.539454 31758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:31.539587 31758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:31.539773 31758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:31.540081 31758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:31.540113 31758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.540136 31758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:31.540181 31758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.545785 31758 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:34501
I20260928 23:09:31.545841 31886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:34501 every 8 connection(s)
I20260928 23:09:31.546168 31758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:31.554585 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.546896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:60347 (local address 127.15.212.126:32799)
0928 23:09:31.546969 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.546971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.547153 (+ 182us) server_negotiation.cc:407] Connection header received
0928 23:09:31.547616 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.547617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.547626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.547678 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:31.548645 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.549138 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.550076 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.550251 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.550303 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.552267 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:31.552268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:31.552269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:31.552275 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:31.553436 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.553839 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.553840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.553840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.553887 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.554094 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.554094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.554094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.554190 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:31.554240 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.554458 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.554521 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:09:31.554809 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.546858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:60347)
0928 23:09:31.547013 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.547054 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.547534 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.547741 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.547744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.548302 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:31.548568 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.548574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.549219 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.549220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.550000 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.550002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.550320 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.550321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.550418 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.550864 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:31.550872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:31.552186 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:31.553504 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.553510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.553511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.553772 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.553946 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.553946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.553947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.554040 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.554244 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:31.554248 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:31.554353 (+ 105us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.554432 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.554545 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260928 23:09:31.554945 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31758
I20260928 23:09:31.555048 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:09:31.555261 16209 external_mini_cluster.cc:1428] Setting key fdee91791b21adf8d6807c74aa47b461
I20260928 23:09:31.555373 31887 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:31.555470 31887 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:31.555685 31887 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:31.556047 31562 ts_manager.cc:194] Registered new tserver with Master: 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501)
I20260928 23:09:31.556481 31562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:60347
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:09:31.579830 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:31.658700 31892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:31.658886 31892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:31.658908 31892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:31.660109 31892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:31.660158 31892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:31.660185 31892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:31.661674 31892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:31.661907 31892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:31.662128 31892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:31.667059 31900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:31.667069 31898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:31.667066 31897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:31.667096 31892 server_base.cc:1034] running on GCE node
I20260928 23:09:31.667514 31892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:31.667776 31892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:31.668921 31892 hybrid_clock.cc:648] HybridClock initialized: now 1790636971668911 us; error 26 us; skew 500 ppm
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:31.671378 31892 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:31.671768 31892 webserver.cc:460] Webserver started at http://127.15.212.67:41629/ using document root <none> and password file <none>
I20260928 23:09:31.671947 31892 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:31.671996 31892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:31.672091 31892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:09:31.673115 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "33ee0603ce314f0786172e762c482217"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "aafae1c204d27c2dacbc5fb5e70917eb"
server_key_iv: "d0634545f1e365c71b9b2afbecdf5d01"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.673414 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "33ee0603ce314f0786172e762c482217"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "aafae1c204d27c2dacbc5fb5e70917eb"
server_key_iv: "d0634545f1e365c71b9b2afbecdf5d01"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.674662 31892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:09:31.675328 31907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.675496 31892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:31.675576 31892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "33ee0603ce314f0786172e762c482217"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "aafae1c204d27c2dacbc5fb5e70917eb"
server_key_iv: "d0634545f1e365c71b9b2afbecdf5d01"
server_key_version: "encryptionkey@0"
I20260928 23:09:31.675628 31892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:31.688948 31892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:31.690639 31892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:31.690738 31892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:31.690891 31892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:31.691121 31892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:31.691150 31892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.691170 31892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:31.691183 31892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:31.696858 31892 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41823
I20260928 23:09:31.696941 32020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41823 every 8 connection(s)
I20260928 23:09:31.697257 31892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:09:31 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:31.704175 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 31892
I20260928 23:09:31.704275 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:09:31.704512 16209 external_mini_cluster.cc:1428] Setting key 80d0cbe82ef856078696759fcd233dc1
I20260928 23:09:31.706391 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.698289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:37251 (local address 127.15.212.126:32799)
0928 23:09:31.698351 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.698353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.698362 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:09:31.698797 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.698798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.698808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.698858 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:31.699707 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.700210 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.701215 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.701426 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.701496 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.703440 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:31.703442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:31.703442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:31.703449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:31.705020 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.705490 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.705491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.705491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.705550 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.705816 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.705817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.705818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.705960 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:31.706033 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.706268 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.706330 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:09:31.706527 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.698035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:37251)
0928 23:09:31.698189 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.698223 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.698715 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.698907 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.698910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.699444 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:31.699634 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.699639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.700286 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.700287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.701142 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.701145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.701499 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.701500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.701615 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.702044 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:31.702052 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:31.703356 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:31.705110 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.705117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.705119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.705416 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.705633 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:31.705634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:31.705634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:31.705746 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:31.706033 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:31.706038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:31.706147 (+ 109us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.706232 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.706342 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260928 23:09:31.706907 32021 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:31.706977 32021 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:31.707103 32021 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:31.707403 31562 ts_manager.cc:194] Registered new tserver with Master: 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:31.707811 31562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:37251
I20260928 23:09:31.716528 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:31.723410 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.717940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.15.212.126:32799)
0928 23:09:31.717996 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.717997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.718089 (+ 92us) server_negotiation.cc:407] Connection header received
0928 23:09:31.718137 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.718137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.718145 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.718200 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:31.718685 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.719126 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.719794 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.719961 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.720009 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.721112 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:31.721113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:31.721113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:31.721119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:31.722540 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.722841 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.722841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.722842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.722890 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:31.723031 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:31.723031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:31.723032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:31.723149 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:31.723216 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.723302 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.723352 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:09:31.724983 31562 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:09:31.726523 31562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:09:31.732859 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.728850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41823 (local address 127.0.0.1:55984)
0928 23:09:31.729361 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.729370 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.729422 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.729707 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.729710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.729713 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:31.729832 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.729835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.730679 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.730680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.731795 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.731798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.732551 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.732552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.732631 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.732643 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.732735 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.732787 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":46,"thread_start_us":96,"threads_started":1}
I20260928 23:09:31.732872 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.729082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.15.212.67:41823)
0928 23:09:31.729234 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.729238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.729435 (+ 197us) server_negotiation.cc:407] Connection header received
0928 23:09:31.729481 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.729483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.729535 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.729620 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:31.729894 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.730562 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.731884 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.732421 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.732538 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.732603 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.732752 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.732796 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260928 23:09:31.733541 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.728850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:44051 (local address 127.0.0.1:49310)
0928 23:09:31.729489 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.729497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.729538 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.729786 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.729786 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.729788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:31.729825 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.729826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.731115 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.731116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.731907 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.731909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.733290 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.733291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.733358 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.733364 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.733422 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.733465 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260928 23:09:31.733600 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.729258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.15.212.65:44051)
0928 23:09:31.729400 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.729422 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.729594 (+ 172us) server_negotiation.cc:407] Connection header received
0928 23:09:31.729633 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.729635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.729648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.729711 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:31.729901 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.731001 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.731996 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.732765 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.732829 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.732881 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.733452 (+ 571us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.733509 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260928 23:09:31.733554 31955 tablet_service.cc:1467] Processing CreateTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a (DEFAULT_TABLE table=test-table [id=48c440e3199340b78631e0874fbfa643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:31.733714 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.728845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:34501 (local address 127.0.0.1:50080)
0928 23:09:31.729151 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.729193 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.729259 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.729721 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.729722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.729724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:31.729799 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.729801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.731106 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.731107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.732503 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.732505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.733468 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.733469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.733533 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.733536 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.733597 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.733641 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":28,"thread_start_us":40,"threads_started":1}
I20260928 23:09:31.733937 31955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13004baf646f4403bf46aba8bcd8fc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:31.734040 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.728926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.15.212.66:34501)
0928 23:09:31.729483 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.729488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.729500 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:09:31.729543 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.729545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.729561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.729636 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:31.729882 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.730998 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.732611 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.733316 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.733383 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.733438 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.733914 (+ 476us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.733963 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":123,"threads_started":1}
I20260928 23:09:31.734408 31687 tablet_service.cc:1467] Processing CreateTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a (DEFAULT_TABLE table=test-table [id=48c440e3199340b78631e0874fbfa643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:31.734647 31687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13004baf646f4403bf46aba8bcd8fc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:31.736516 32038 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Bootstrap starting.
I20260928 23:09:31.737035 31821 tablet_service.cc:1467] Processing CreateTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a (DEFAULT_TABLE table=test-table [id=48c440e3199340b78631e0874fbfa643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:31.737356 31821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13004baf646f4403bf46aba8bcd8fc2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:31.737363 32038 tablet_bootstrap.cc:654] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:31.737816 32038 log.cc:824] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:31.738166 32039 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap starting.
I20260928 23:09:31.738518 32038 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: No bootstrap required, opened a new log
I20260928 23:09:31.738579 32038 ts_tablet_manager.cc:1397] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:09:31.739871 32041 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Bootstrap starting.
I20260928 23:09:31.739972 32038 raft_consensus.cc:348] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.740177 32038 raft_consensus.cc:374] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:31.740221 32038 raft_consensus.cc:729] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ee0603ce314f0786172e762c482217, State: Initialized, Role: FOLLOWER
I20260928 23:09:31.740357 32038 consensus_queue.cc:260] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.740556 32041 tablet_bootstrap.cc:654] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:31.740669 32038 ts_tablet_manager.cc:1428] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:09:31.740801 32021 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
I20260928 23:09:31.741048 32039 tablet_bootstrap.cc:654] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:31.741168 32041 log.cc:824] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:31.741605 32039 log.cc:824] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:31.741955 32041 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: No bootstrap required, opened a new log
I20260928 23:09:31.742034 32041 ts_tablet_manager.cc:1397] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:09:31.742321 32039 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: No bootstrap required, opened a new log
I20260928 23:09:31.742372 32039 ts_tablet_manager.cc:1397] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 23:09:31.743395 32041 raft_consensus.cc:348] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.743548 32041 raft_consensus.cc:374] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:31.743595 32041 raft_consensus.cc:729] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6751b5727f54403da9d44a8cc4e52ca7, State: Initialized, Role: FOLLOWER
I20260928 23:09:31.743712 32041 consensus_queue.cc:260] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.743755 32039 raft_consensus.cc:348] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.743939 32039 raft_consensus.cc:374] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:31.743974 32041 ts_tablet_manager.cc:1428] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:09:31.744001 32039 raft_consensus.cc:729] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3044f9c64c64e3a8aaa147b44c83b0c, State: Initialized, Role: FOLLOWER
I20260928 23:09:31.744014 31887 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
I20260928 23:09:31.744118 32039 consensus_queue.cc:260] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.744450 31753 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
I20260928 23:09:31.744493 32039 ts_tablet_manager.cc:1428] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260928 23:09:31.796722 31888 tablet.cc:2369] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:09:31.889995 31754 tablet.cc:2369] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:09:31.947953 32022 tablet.cc:2369] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:09:31.958734 32045 raft_consensus.cc:482] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:31.958868 32045 raft_consensus.cc:504] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.959267 32045 leader_election.cc:288] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:31.962023 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.959395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:44051 (local address 127.15.212.66:37251)
0928 23:09:31.959441 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.959455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.959519 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.959700 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.959701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.959704 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:31.959775 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.959777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.960458 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.960459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.961246 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.961248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.961765 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.961766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.961846 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.961849 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.961912 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.961948 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260928 23:09:31.962021 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.959478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:37251 (local address 127.15.212.65:44051)
0928 23:09:31.959531 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.959533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.959545 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:09:31.959588 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.959589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.959598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.959681 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:31.959842 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.960346 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.961339 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.961679 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.961758 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.961766 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.961917 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.961951 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 23:09:31.962321 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.959442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41823 (local address 127.15.212.66:34385)
0928 23:09:31.959651 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:31.959668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:31.959740 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:31.959876 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:31.959877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:31.959878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:31.959988 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.959989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.960699 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.960699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.961533 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:31.961535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.962079 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.962080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:31.962152 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.962156 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:31.962204 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:09:31.962238 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":123,"threads_started":1}
I20260928 23:09:31.962383 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:31.959570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:34385 (local address 127.15.212.67:41823)
0928 23:09:31.959630 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:31.959632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:31.959730 (+ 98us) server_negotiation.cc:407] Connection header received
0928 23:09:31.959789 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:31.959790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:31.959799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:31.959874 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:31.960052 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.960604 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.961619 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:31.961996 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:31.962079 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:31.962086 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:31.962276 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 23:09:31.962321 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:09:31.962371 31707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13004baf646f4403bf46aba8bcd8fc2a" candidate_uuid: "6751b5727f54403da9d44a8cc4e52ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" is_pre_election: true
I20260928 23:09:31.962518 31707 raft_consensus.cc:2454] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6751b5727f54403da9d44a8cc4e52ca7 in term 0.
I20260928 23:09:31.962634 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13004baf646f4403bf46aba8bcd8fc2a" candidate_uuid: "6751b5727f54403da9d44a8cc4e52ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33ee0603ce314f0786172e762c482217" is_pre_election: true
I20260928 23:09:31.962816 31975 raft_consensus.cc:2454] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6751b5727f54403da9d44a8cc4e52ca7 in term 0.
I20260928 23:09:31.962872 31775 leader_election.cc:302] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6751b5727f54403da9d44a8cc4e52ca7, d3044f9c64c64e3a8aaa147b44c83b0c; no voters:
I20260928 23:09:31.962987 32045 raft_consensus.cc:2781] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:31.963060 32045 raft_consensus.cc:482] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:31.963090 32045 raft_consensus.cc:3037] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:31.963742 32045 raft_consensus.cc:504] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.963871 32045 leader_election.cc:288] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [CANDIDATE]: Term 1 election: Requested vote from peers d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:31.964002 31707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13004baf646f4403bf46aba8bcd8fc2a" candidate_uuid: "6751b5727f54403da9d44a8cc4e52ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
I20260928 23:09:31.964016 31975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13004baf646f4403bf46aba8bcd8fc2a" candidate_uuid: "6751b5727f54403da9d44a8cc4e52ca7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33ee0603ce314f0786172e762c482217"
I20260928 23:09:31.964078 31975 raft_consensus.cc:3037] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:31.964078 31707 raft_consensus.cc:3037] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:31.964949 31707 raft_consensus.cc:2454] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6751b5727f54403da9d44a8cc4e52ca7 in term 1.
I20260928 23:09:31.964949 31975 raft_consensus.cc:2454] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6751b5727f54403da9d44a8cc4e52ca7 in term 1.
I20260928 23:09:31.965104 31775 leader_election.cc:302] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6751b5727f54403da9d44a8cc4e52ca7, d3044f9c64c64e3a8aaa147b44c83b0c; no voters:
I20260928 23:09:31.965201 32045 raft_consensus.cc:2781] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:31.965359 32045 raft_consensus.cc:686] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 LEADER]: Becoming Leader. State: Replica: 6751b5727f54403da9d44a8cc4e52ca7, State: Running, Role: LEADER
I20260928 23:09:31.965461 32045 consensus_queue.cc:237] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:31.966300 31561 catalog_manager.cc:5495] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66). New cstate: current_term: 1 leader_uuid: "6751b5727f54403da9d44a8cc4e52ca7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } health_report { overall_health: UNKNOWN } } }
I20260928 23:09:32.013566 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:32.011106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.15.212.66:34501)
0928 23:09:32.011150 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:32.011152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:32.011160 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:32.011200 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:32.011203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:32.011224 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:32.011298 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:32.011474 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:32.011953 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:32.012748 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:32.012951 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:32.013027 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:32.013223 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:32.013319 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:32.013414 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:32.013454 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 23:09:32.013473 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:09:32.015327 31707 raft_consensus.cc:1261] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Refusing update from remote peer 6751b5727f54403da9d44a8cc4e52ca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:32.015337 31975 raft_consensus.cc:1261] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Refusing update from remote peer 6751b5727f54403da9d44a8cc4e52ca7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:32.015671 32045 consensus_queue.cc:1035] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:32.015821 32048 consensus_queue.cc:1035] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:32.018664 32053 mvcc.cc:205] Tried to move back new op lower bound from 7334449037372092416 to 7334449037171159040. Current Snapshot: MvccSnapshot[applied={T|T < 7334449037372092416}]
I20260928 23:09:32.020939 31561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:32.021018 31561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:32.021749 31561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Sending DeleteTablet for 3 replicas of tablet 13004baf646f4403bf46aba8bcd8fc2a
I20260928 23:09:32.022079 31821 tablet_service.cc:1514] Processing DeleteTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:32 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:50080
I20260928 23:09:32.022080 31687 tablet_service.cc:1514] Processing DeleteTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:32 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:49310
I20260928 23:09:32.022133 31955 tablet_service.cc:1514] Processing DeleteTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:32 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:55984
I20260928 23:09:32.022352 32062 tablet_replica.cc:330] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: stopping tablet replica
I20260928 23:09:32.022375 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31624
I20260928 23:09:32.022455 32062 raft_consensus.cc:2229] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:32.022564 32062 raft_consensus.cc:2258] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:32.022702 32063 tablet_replica.cc:330] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: stopping tablet replica
I20260928 23:09:32.022779 32063 raft_consensus.cc:2229] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:32.022949 32063 raft_consensus.cc:2258] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:32.023649 32061 tablet_replica.cc:330] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: stopping tablet replica
I20260928 23:09:32.023715 32061 raft_consensus.cc:2229] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:09:32.023790 32061 raft_consensus.cc:2258] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:32.024164 32061 ts_tablet_manager.cc:1905] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:32.025767 32061 ts_tablet_manager.cc:1918] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:32.025817 32061 log.cc:1196] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/13004baf646f4403bf46aba8bcd8fc2a
I20260928 23:09:32.026034 32061 ts_tablet_manager.cc:1939] T 13004baf646f4403bf46aba8bcd8fc2a P 6751b5727f54403da9d44a8cc4e52ca7: Deleting consensus metadata
I20260928 23:09:32.028721 31548 catalog_manager.cc:4841] TS 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): tablet 13004baf646f4403bf46aba8bcd8fc2a (table test-table [id=48c440e3199340b78631e0874fbfa643]) successfully deleted
W20260928 23:09:32.029093 31547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:44051 (error 108)
I20260928 23:09:32.029228 32063 ts_tablet_manager.cc:1905] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:32.029538 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:32.029430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:44051 (local address 127.0.0.1:49318)
0928 23:09:32.029472 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:32.029508 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260928 23:09:32.029659 31547 catalog_manager.cc:4584] TS d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13004baf646f4403bf46aba8bcd8fc2a: Network error: Client connection negotiation failed: client connection to 127.15.212.65:44051: connect: Connection refused (error 111)
I20260928 23:09:32.029953 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31758
I20260928 23:09:32.031840 32063 ts_tablet_manager.cc:1918] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:32.031913 32063 log.cc:1196] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/13004baf646f4403bf46aba8bcd8fc2a
I20260928 23:09:32.032196 32063 ts_tablet_manager.cc:1939] T 13004baf646f4403bf46aba8bcd8fc2a P 33ee0603ce314f0786172e762c482217: Deleting consensus metadata
I20260928 23:09:32.032622 31549 catalog_manager.cc:4841] TS 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): tablet 13004baf646f4403bf46aba8bcd8fc2a (table test-table [id=48c440e3199340b78631e0874fbfa643]) successfully deleted
I20260928 23:09:32.035176 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31892
I20260928 23:09:32.039366 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 31530
I20260928 23:09:52.044500 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--webserver_interface=127.15.212.126
--webserver_port=37457
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:32799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:09:52.128757 32065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:52.129048 32065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:52.129089 32065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:52.131068 32065 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 23:09:52.131130 32065 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 23:09:52.131145 32065 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 23:09:52.131157 32065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:09:52.131202 32065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:52.131227 32065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:52.131247 32065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:09:52.131278 32065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:09:52.133342 32065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:32799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=37457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:52.133646 32065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:52.133891 32065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:52.142925 32071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.142979 32072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.143028 32065 server_base.cc:1034] running on GCE node
W20260928 23:09:52.143013 32074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.143432 32065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:52.143826 32065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:52.145008 32065 hybrid_clock.cc:648] HybridClock initialized: now 1790636992144998 us; error 43 us; skew 500 ppm
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:52.147630 32065 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:09:52.148128 32065 webserver.cc:460] Webserver started at http://127.15.212.126:37457/ using document root <none> and password file <none>
I20260928 23:09:52.148380 32065 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:52.148484 32065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:52.149902 32065 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:52.150677 32081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.150852 32065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:52.150960 32065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "6046e2d01d204bb59888504e00ea65ba"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "8a173a666488bdcd430a4fb22f419b56"
server_key_iv: "7db8b8b288bbe49d5a7c7c506f900e74"
server_key_version: "encryptionkey@0"
I20260928 23:09:52.151367 32065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:52.164160 32065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:52.166127 32065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:52.166286 32065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:52.170486 32065 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:32799
I20260928 23:09:52.170519 32133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:32799 every 8 connection(s)
I20260928 23:09:52.170921 32065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:09:52.172341 32134 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:09:52.173045 32134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap starting.
I20260928 23:09:52.179577 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32065
I20260928 23:09:52.179924 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:44051
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=36043
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:52.181821 32134 log.cc:824] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:52.183854 32134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:52.184288 32134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap complete.
I20260928 23:09:52.186605 32134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:52.186822 32134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Initialized, Role: FOLLOWER
I20260928 23:09:52.187000 32134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:52.187059 32134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:09:52.187126 32134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:09:52.187170 32134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 1 FOLLOWER]: Advancing to term 2
I20260928 23:09:52.188187 32134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:52.188271 32134 leader_election.cc:302] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6046e2d01d204bb59888504e00ea65ba; no voters:
I20260928 23:09:52.188482 32134 leader_election.cc:288] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 23:09:52.188517 32138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Leader election won for term 2
I20260928 23:09:52.188681 32138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 LEADER]: Becoming Leader. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Running, Role: LEADER
I20260928 23:09:52.188766 32134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:09:52.188820 32138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:09:52.189091 32138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6046e2d01d204bb59888504e00ea65ba. Latest consensus state: current_term: 2 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:09:52.189163 32138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:52.189301 32139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:09:52.189374 32139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
I20260928 23:09:52.189697 32143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:09:52.190572 32143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:09:52.190692 32143 catalog_manager.cc:1259] Loaded cluster ID: 90f66fa3b0eb4697aa59edfef23d3064
I20260928 23:09:52.190912 32143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:09:52.191713 32143 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:09:52.191906 32143 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Loaded TSK: 0
I20260928 23:09:52.199193 32143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Generated new TSK 1
I20260928 23:09:52.199296 32143 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 23:09:52.280687 32136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:52.280879 32136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:52.280901 32136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:52.282052 32136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:52.282094 32136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:52.282120 32136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:09:52.283535 32136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:44051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=36043
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:52.283782 32136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:52.284029 32136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:52.289860 32164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.289868 32161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.289860 32162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.290154 32136 server_base.cc:1034] running on GCE node
I20260928 23:09:52.290401 32136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:52.290675 32136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:52.291836 32136 hybrid_clock.cc:648] HybridClock initialized: now 1790636992291823 us; error 28 us; skew 500 ppm
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:52.294502 32136 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:09:52.295044 32136 webserver.cc:460] Webserver started at http://127.15.212.65:36043/ using document root <none> and password file <none>
I20260928 23:09:52.295301 32136 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:52.295387 32136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:52.297220 32136 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:09:52.298045 32171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.298269 32136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:52.298365 32136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d5dd92f1bf8c0d5959507e11e7edc334"
server_key_iv: "77296a8942ff3e442fb52c6422a0b43a"
server_key_version: "encryptionkey@0"
I20260928 23:09:52.298817 32136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:52.314563 32136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:52.317045 32136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:52.317198 32136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:52.317510 32136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:52.318076 32178 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 23:09:52.319535 32136 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 23:09:52.319587 32136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 23:09:52.319624 32136 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 23:09:52.320326 32136 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 23:09:52.320369 32136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:52.320417 32178 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap starting.
I20260928 23:09:52.328204 32136 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:44051
I20260928 23:09:52.328799 32136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:09:52.331820 32178 log.cc:824] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:52.332401 32285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:44051 every 8 connection(s)
I20260928 23:09:52.334424 32178 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:09:52.334764 32178 tablet_bootstrap.cc:492] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap complete.
I20260928 23:09:52.335556 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32136
I20260928 23:09:52.335865 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:34501
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=34735
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:09:52.335865 32178 ts_tablet_manager.cc:1397] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260928 23:09:52.337694 32178 raft_consensus.cc:348] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:52.338048 32178 raft_consensus.cc:729] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3044f9c64c64e3a8aaa147b44c83b0c, State: Initialized, Role: FOLLOWER
I20260928 23:09:52.338210 32178 consensus_queue.cc:260] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } } peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } }
I20260928 23:09:52.338502 32178 ts_tablet_manager.cc:1428] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent starting tablet: real 0.003s user 0.003s sys 0.002s
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:52.346527 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.329691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:36033)
0928 23:09:52.332751 (+ 3060us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.332798 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.333481 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.334147 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.334151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.334511 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:52.334807 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.334814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.336794 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.336795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.338389 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.338392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.339358 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.339359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.339516 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.340183 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:52.340203 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:52.342096 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:52.344614 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.344621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.344625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.344957 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.345274 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.345275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.345276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.345402 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.345907 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:52.345914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:52.346073 (+ 159us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.346181 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.346337 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3013,"thread_start_us":114,"threads_started":1}
I20260928 23:09:52.347033 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.329765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:36033 (local address 127.15.212.126:32799)
0928 23:09:52.333696 (+ 3931us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.333706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.333747 (+ 41us) server_negotiation.cc:407] Connection header received
0928 23:09:52.333876 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.333881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.333935 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.334073 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.334922 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.336672 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.338497 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.339253 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.339367 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.342215 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.342220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.342221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.342234 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.344502 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.345055 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.345058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.345059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.345190 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.345488 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.345489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.345489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.345819 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.345919 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.346268 (+ 349us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.346426 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3876,"thread_start_us":82,"threads_started":1}
I20260928 23:09:52.347754 32286 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:52.347856 32286 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:52.348043 32286 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:52.348834 32098 ts_manager.cc:194] Registered new tserver with Master: d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051)
I20260928 23:09:52.349982 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:36033
I20260928 23:09:52.350942 32286 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.65@KRBTEST.COM
I20260928 23:09:52.357666 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.349539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.15.212.65:44051)
0928 23:09:52.349714 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.349719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.349748 (+ 29us) server_negotiation.cc:407] Connection header received
0928 23:09:52.349903 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.349905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.349921 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.349978 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.350528 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.351563 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.352319 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.352696 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.352777 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.354807 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.354813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.354814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.354821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.356342 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.356803 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.356806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.356806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.356865 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.357074 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.357075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.357075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.357208 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.357294 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.357488 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.357576 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260928 23:09:52.357666 32293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.349436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:44051 (local address 127.0.0.1:53684)
0928 23:09:52.349659 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.349694 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.349765 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.349996 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.350001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.350359 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:52.350440 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.350442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.351655 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.351656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.352216 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.352218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.352776 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.352777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.352856 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.353340 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:52.353345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:52.354696 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:52.356499 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.356505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.356507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.356699 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.356934 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.356935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.356935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.357000 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.357286 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:52.357292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:52.357341 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.357458 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.357572 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260928 23:09:52.358104 32202 tablet_service.cc:1514] Processing DeleteTablet for tablet 13004baf646f4403bf46aba8bcd8fc2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:32 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53684
I20260928 23:09:52.358669 32295 tablet_replica.cc:330] stopping tablet replica
I20260928 23:09:52.358780 32295 raft_consensus.cc:2229] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:52.358867 32295 raft_consensus.cc:2258] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:52.359201 32295 ts_tablet_manager.cc:1905] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:52.361728 32295 ts_tablet_manager.cc:1918] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:52.361827 32295 log.cc:1196] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal/wals/13004baf646f4403bf46aba8bcd8fc2a
I20260928 23:09:52.362047 32295 ts_tablet_manager.cc:1939] T 13004baf646f4403bf46aba8bcd8fc2a P d3044f9c64c64e3a8aaa147b44c83b0c: Deleting consensus metadata
I20260928 23:09:52.362524 32083 catalog_manager.cc:4841] TS d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051): tablet 13004baf646f4403bf46aba8bcd8fc2a (table test-table [id=48c440e3199340b78631e0874fbfa643]) successfully deleted
W20260928 23:09:52.423069 32291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:52.423275 32291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:52.423298 32291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:52.424525 32291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:52.424566 32291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:52.424592 32291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:09:52.426034 32291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:34501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=34735
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:52.426291 32291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:52.426544 32291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:52.431591 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.431663 32303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.431656 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.431838 32291 server_base.cc:1034] running on GCE node
I20260928 23:09:52.432057 32291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:52.432309 32291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:52.433473 32291 hybrid_clock.cc:648] HybridClock initialized: now 1790636992433475 us; error 49 us; skew 500 ppm
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:52.436066 32291 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:09:52.436583 32291 webserver.cc:460] Webserver started at http://127.15.212.66:34735/ using document root <none> and password file <none>
I20260928 23:09:52.436770 32291 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:52.436823 32291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:52.438136 32291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:52.438701 32310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.438885 32291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:09:52.438969 32291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "6751b5727f54403da9d44a8cc4e52ca7"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d7c4bb53310b87d2fcaa565e806d9e4b"
server_key_iv: "c9d34db19278c1e6967dbd16db9cf78a"
server_key_version: "encryptionkey@0"
I20260928 23:09:52.439289 32291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:52.457377 32291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:52.459830 32291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:52.459951 32291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:52.460148 32291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:52.460568 32291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:52.460610 32291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.460661 32291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:52.460690 32291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.466527 32291 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:34501
I20260928 23:09:52.466614 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:34501 every 8 connection(s)
I20260928 23:09:52.466959 32291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:09:52.471769 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32291
I20260928 23:09:52.472438 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:41823
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=41629
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:52.477211 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.467785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:43803 (local address 127.15.212.126:32799)
0928 23:09:52.467859 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.467860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.468075 (+ 215us) server_negotiation.cc:407] Connection header received
0928 23:09:52.468667 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.468668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.468685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.468742 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.469664 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.470163 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.471143 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.471341 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.471439 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.474189 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.474190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.474191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.474198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.475617 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.476170 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.476171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.476172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.476223 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.476555 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.476556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.476556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.476663 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.476717 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.477075 (+ 358us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.477147 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:09:52.477563 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.467779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:43803)
0928 23:09:52.467943 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.467979 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.468556 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.468798 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.468801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.469366 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:52.469570 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.469575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.470256 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.470256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.471060 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.471062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.471496 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.471497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.471702 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.472307 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:52.472318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:52.474089 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:52.475744 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.475752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.475754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.476089 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.476294 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.476295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.476296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.476481 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.476732 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:52.476739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:52.476923 (+ 184us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.477045 (+ 122us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.477241 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260928 23:09:52.478224 32424 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:52.478338 32424 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:52.478570 32424 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:52.479007 32098 ts_manager.cc:194] Registered new tserver with Master: 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501)
I20260928 23:09:52.479413 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:43803
W20260928 23:09:52.555801 32427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:09:52.556048 32427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:09:52.556089 32427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:09:52.557238 32427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:09:52.557307 32427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:09:52.557351 32427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:09:52.559037 32427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:41823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=41629
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:09:52.559288 32427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:09:52.559530 32427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:09:52.564698 32432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:09:52.564754 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.564815 32427 server_base.cc:1034] running on GCE node
W20260928 23:09:52.564705 32433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:09:52.565187 32427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:09:52.565385 32427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:09:52.566521 32427 hybrid_clock.cc:648] HybridClock initialized: now 1790636992566512 us; error 30 us; skew 500 ppm
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:09:52.568980 32427 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:09:52.569432 32427 webserver.cc:460] Webserver started at http://127.15.212.67:41629/ using document root <none> and password file <none>
I20260928 23:09:52.569617 32427 fs_manager.cc:359] Metadata directory not provided
I20260928 23:09:52.569669 32427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:09:52.570910 32427 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:09:52.571446 32442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.571624 32427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.571730 32427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "33ee0603ce314f0786172e762c482217"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "aafae1c204d27c2dacbc5fb5e70917eb"
server_key_iv: "d0634545f1e365c71b9b2afbecdf5d01"
server_key_version: "encryptionkey@0"
I20260928 23:09:52.572072 32427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:09:52.581508 32427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:09:52.583416 32427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:09:52.583585 32427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:09:52.583820 32427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:09:52.584141 32427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:09:52.584195 32427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.584239 32427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:09:52.584266 32427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:09:52.590488 32427 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41823
I20260928 23:09:52.590530 32555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41823 every 8 connection(s)
I20260928 23:09:52.590880 32427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:09:52 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790636992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:09:52.597482 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32427
I20260928 23:09:52.600618 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.591597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:55283 (local address 127.15.212.126:32799)
0928 23:09:52.591672 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.591674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.591894 (+ 220us) server_negotiation.cc:407] Connection header received
0928 23:09:52.592420 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.592421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.592447 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.592511 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.593485 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.593981 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.594962 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.595154 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.595217 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.597329 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.597331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.597332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.597341 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.599219 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.599595 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.599596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.599597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.599672 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.599945 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.599945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.599946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.600061 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.600139 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.600415 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.600526 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:09:52.600823 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.591600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:55283)
0928 23:09:52.591764 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.591805 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.592279 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.592591 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.592595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.593183 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:09:52.593385 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.593390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.594078 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.594079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.594858 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.594860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.595236 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.595237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.595336 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.595807 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:09:52.595814 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:09:52.597229 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:09:52.599300 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.599305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.599307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.599524 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.599754 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:09:52.599755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:09:52.599756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:09:52.599875 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:09:52.600131 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:09:52.600137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:09:52.600282 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.600394 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.600572 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260928 23:09:52.601290 32556 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32799
I20260928 23:09:52.601416 32556 heartbeater.cc:464] Registering TS with master...
I20260928 23:09:52.601615 32556 heartbeater.cc:511] Master 127.15.212.126:32799 requested a full tablet report, sending...
I20260928 23:09:52.602100 32098 ts_manager.cc:194] Registered new tserver with Master: 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:52.602586 32098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:55283
I20260928 23:09:52.604889 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.597935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.15.212.126:32799)
0928 23:09:52.598054 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.598056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.598176 (+ 120us) server_negotiation.cc:407] Connection header received
0928 23:09:52.598226 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.598226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.598238 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.598306 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.598870 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.599578 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.600422 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.600725 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.600802 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.602319 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.602320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.602321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.602327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.603818 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.604137 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.604138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.604138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.604186 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.604345 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.604345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.604345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.604476 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.604663 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.604702 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.604786 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260928 23:09:52.605657 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:09:52.608098 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.606012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.15.212.126:32799)
0928 23:09:52.606048 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.606049 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.606140 (+ 91us) server_negotiation.cc:407] Connection header received
0928 23:09:52.606185 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.606186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.606213 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.606271 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:52.606362 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.606810 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.607484 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.607663 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.607712 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.607893 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:52.607932 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 23:09:52.608059 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47940: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 23:09:52.608161 32560 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47940: token expired
W20260928 23:09:52.608289 32027 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.212.126:32799)
I20260928 23:09:52.613969 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.608643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.15.212.126:32799)
0928 23:09:52.608675 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.608677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.608684 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:09:52.608743 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.608743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.608750 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.608798 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:09:52.609254 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.609677 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.610359 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.610522 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.610568 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.611744 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:09:52.611746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:09:52.611746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:09:52.611752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:09:52.613109 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.613411 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.613412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.613412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.613459 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:09:52.613617 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:09:52.613618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:09:52.613618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:09:52.613704 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:09:52.613774 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.613851 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.613904 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260928 23:09:52.615186 32098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:09:52.616894 32098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:09:52.619223 32202 tablet_service.cc:1467] Processing CreateTablet for tablet 879310b172884df3bf64c9655253ec4b (DEFAULT_TABLE table=test-table [id=64a1ac37cf7a4f468fbedaa427ec54bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:52.619421 32202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879310b172884df3bf64c9655253ec4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:52.620990 32178 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap starting.
I20260928 23:09:52.621289 32178 tablet_bootstrap.cc:654] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:52.621909 32178 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: No bootstrap required, opened a new log
I20260928 23:09:52.621963 32178 ts_tablet_manager.cc:1397] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 23:09:52.622304 32178 raft_consensus.cc:348] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.622390 32178 raft_consensus.cc:374] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:52.622473 32178 raft_consensus.cc:729] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3044f9c64c64e3a8aaa147b44c83b0c, State: Initialized, Role: FOLLOWER
I20260928 23:09:52.622534 32178 consensus_queue.cc:260] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.622628 32178 ts_tablet_manager.cc:1428] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 23:09:52.623293 32293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.618835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41823 (local address 127.0.0.1:59812)
0928 23:09:52.618869 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.618880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.618917 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.619448 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.619450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.619452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:52.619493 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.619495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.620913 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.620914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.622086 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.622088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.623025 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.623026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.623095 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.623118 (+ 23us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.623174 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.623221 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":29}
I20260928 23:09:52.623307 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.619025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.15.212.67:41823)
0928 23:09:52.619283 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.619287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.619297 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:09:52.619333 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.619335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.619353 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.619436 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:52.619560 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.620827 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.622151 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.622871 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.622939 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.622994 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.623190 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.623236 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260928 23:09:52.623952 32490 tablet_service.cc:1467] Processing CreateTablet for tablet 879310b172884df3bf64c9655253ec4b (DEFAULT_TABLE table=test-table [id=64a1ac37cf7a4f468fbedaa427ec54bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:52.624163 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.618835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:34501 (local address 127.0.0.1:46494)
0928 23:09:52.619040 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.619049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.619088 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.619519 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.619520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.619522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:52.619565 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.619566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.620938 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.620939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.623107 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.623109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.623933 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.623934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.623995 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.623999 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.624057 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.624094 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":15,"thread_start_us":141,"threads_started":1}
I20260928 23:09:52.624183 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.619093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.15.212.66:34501)
0928 23:09:52.619240 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.619245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.619257 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:09:52.619302 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.619305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.619322 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.619415 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:52.619647 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.620845 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.623202 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.623821 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.623926 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.623983 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.624071 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.624106 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260928 23:09:52.624326 32490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879310b172884df3bf64c9655253ec4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:52.624930 32358 tablet_service.cc:1467] Processing CreateTablet for tablet 879310b172884df3bf64c9655253ec4b (DEFAULT_TABLE table=test-table [id=64a1ac37cf7a4f468fbedaa427ec54bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:09:52.625232 32358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879310b172884df3bf64c9655253ec4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:09:52.627038 32566 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Bootstrap starting.
I20260928 23:09:52.627705 32566 tablet_bootstrap.cc:654] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:52.628096 32567 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Bootstrap starting.
I20260928 23:09:52.628336 32566 log.cc:824] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:52.628784 32567 tablet_bootstrap.cc:654] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Neither blocks nor log segments found. Creating new log.
I20260928 23:09:52.629252 32566 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: No bootstrap required, opened a new log
I20260928 23:09:52.629333 32566 ts_tablet_manager.cc:1397] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:09:52.629375 32567 log.cc:824] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Log is configured to *not* fsync() on all Append() calls
I20260928 23:09:52.630069 32567 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: No bootstrap required, opened a new log
I20260928 23:09:52.630147 32567 ts_tablet_manager.cc:1397] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:09:52.630887 32566 raft_consensus.cc:348] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.631022 32566 raft_consensus.cc:374] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:52.631044 32566 raft_consensus.cc:729] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ee0603ce314f0786172e762c482217, State: Initialized, Role: FOLLOWER
I20260928 23:09:52.631165 32566 consensus_queue.cc:260] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.631449 32566 ts_tablet_manager.cc:1428] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:09:52.631486 32556 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
I20260928 23:09:52.631812 32567 raft_consensus.cc:348] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.631975 32567 raft_consensus.cc:374] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:09:52.632019 32567 raft_consensus.cc:729] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6751b5727f54403da9d44a8cc4e52ca7, State: Initialized, Role: FOLLOWER
I20260928 23:09:52.632138 32567 consensus_queue.cc:260] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.632462 32424 heartbeater.cc:503] Master 127.15.212.126:32799 was elected leader, sending a full tablet report...
I20260928 23:09:52.632685 32567 ts_tablet_manager.cc:1428] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:09:52.665071 32292 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:09:52.665238 32292 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.665657 32292 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:52.668515 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.665795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:34501 (local address 127.15.212.65:55839)
0928 23:09:52.665857 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.665868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.665922 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.666057 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.666057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.666059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:52.666115 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.666117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.666767 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.666768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.667653 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.667656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.668207 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.668208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.668286 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.668304 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.668380 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.668420 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 23:09:52.668515 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.665850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:55839 (local address 127.15.212.66:34501)
0928 23:09:52.665891 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.665892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.665901 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:09:52.665990 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.665991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.666001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.666054 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:52.666213 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.666683 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.667746 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.668126 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.668211 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.668220 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.668390 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.668427 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:09:52.668674 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.665859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:39667 (local address 127.15.212.67:41823)
0928 23:09:52.665920 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.665922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.666193 (+ 271us) server_negotiation.cc:407] Connection header received
0928 23:09:52.666271 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.666272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.666281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.666333 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:09:52.666494 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.666950 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.667877 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.668254 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.668352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.668361 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.668572 (+ 211us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.668611 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:09:52.668678 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.665801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41823 (local address 127.15.212.65:39667)
0928 23:09:52.666112 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.666123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:09:52.666206 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:09:52.666336 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:09:52.666337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:09:52.666338 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:09:52.666422 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.666423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.667049 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.667050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.667779 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:09:52.667781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.668372 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.668373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:09:52.668509 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.668513 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:09:52.668567 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 23:09:52.668605 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":140,"thread_start_us":117,"threads_started":1}
I20260928 23:09:52.668917 32378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879310b172884df3bf64c9655253ec4b" candidate_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6751b5727f54403da9d44a8cc4e52ca7" is_pre_election: true
I20260928 23:09:52.669097 32378 raft_consensus.cc:2454] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3044f9c64c64e3a8aaa147b44c83b0c in term 0.
I20260928 23:09:52.669062 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879310b172884df3bf64c9655253ec4b" candidate_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33ee0603ce314f0786172e762c482217" is_pre_election: true
I20260928 23:09:52.669215 32510 raft_consensus.cc:2454] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3044f9c64c64e3a8aaa147b44c83b0c in term 0.
I20260928 23:09:52.669332 32174 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6751b5727f54403da9d44a8cc4e52ca7, d3044f9c64c64e3a8aaa147b44c83b0c; no voters:
I20260928 23:09:52.669466 32292 raft_consensus.cc:2781] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:09:52.669524 32292 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:09:52.669546 32292 raft_consensus.cc:3037] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:52.670217 32292 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.670356 32292 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 1 election: Requested vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
I20260928 23:09:52.670485 32378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879310b172884df3bf64c9655253ec4b" candidate_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6751b5727f54403da9d44a8cc4e52ca7"
I20260928 23:09:52.670523 32510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879310b172884df3bf64c9655253ec4b" candidate_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33ee0603ce314f0786172e762c482217"
I20260928 23:09:52.670565 32378 raft_consensus.cc:3037] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:52.670594 32510 raft_consensus.cc:3037] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:09:52.671445 32510 raft_consensus.cc:2454] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3044f9c64c64e3a8aaa147b44c83b0c in term 1.
I20260928 23:09:52.671445 32378 raft_consensus.cc:2454] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3044f9c64c64e3a8aaa147b44c83b0c in term 1.
I20260928 23:09:52.671641 32174 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6751b5727f54403da9d44a8cc4e52ca7, d3044f9c64c64e3a8aaa147b44c83b0c; no voters:
I20260928 23:09:52.671808 32292 raft_consensus.cc:2781] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:09:52.671963 32292 raft_consensus.cc:686] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 LEADER]: Becoming Leader. State: Replica: d3044f9c64c64e3a8aaa147b44c83b0c, State: Running, Role: LEADER
I20260928 23:09:52.672021 32292 consensus_queue.cc:237] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:09:52.672794 32098 catalog_manager.cc:5495] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65). New cstate: current_term: 1 leader_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } health_report { overall_health: HEALTHY } } }
I20260928 23:09:52.682200 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.679752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.15.212.65:44051)
0928 23:09:52.679793 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:09:52.679796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:09:52.679804 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:09:52.679858 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:09:52.679859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:09:52.679883 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:09:52.679953 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:09:52.680114 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.680645 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.681383 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:09:52.681612 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:09:52.681687 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:09:52.681873 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:09:52.681965 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:09:52.682044 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 23:09:52.682098 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 23:09:52.682116 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:09:52.683569 32510 raft_consensus.cc:1261] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Refusing update from remote peer d3044f9c64c64e3a8aaa147b44c83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:52.683586 32378 raft_consensus.cc:1261] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Refusing update from remote peer d3044f9c64c64e3a8aaa147b44c83b0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:09:52.683878 32573 consensus_queue.cc:1035] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:52.684029 32292 consensus_queue.cc:1035] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:09:52.686805 32577 mvcc.cc:205] Tried to move back new op lower bound from 7334449122029625344 to 7334449121985245184. Current Snapshot: MvccSnapshot[applied={T|T < 7334449122029625344}]
I20260928 23:09:52.687291 32578 mvcc.cc:205] Tried to move back new op lower bound from 7334449122029625344 to 7334449121985245184. Current Snapshot: MvccSnapshot[applied={T|T < 7334449122029625344}]
I20260928 23:09:52.688277 32097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:52.688374 32097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:09:52.689060 32097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Sending DeleteTablet for 3 replicas of tablet 879310b172884df3bf64c9655253ec4b
I20260928 23:09:52.689404 32490 tablet_service.cc:1514] Processing DeleteTablet for tablet 879310b172884df3bf64c9655253ec4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:52 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:59812
I20260928 23:09:52.689446 32358 tablet_service.cc:1514] Processing DeleteTablet for tablet 879310b172884df3bf64c9655253ec4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:09:52 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:46494
I20260928 23:09:52.689770 32587 tablet_replica.cc:330] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: stopping tablet replica
I20260928 23:09:52.689774 32586 tablet_replica.cc:330] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: stopping tablet replica
I20260928 23:09:52.689858 32587 raft_consensus.cc:2229] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:52.689927 32586 raft_consensus.cc:2229] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:09:52.689987 32587 raft_consensus.cc:2258] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:52.690088 32586 raft_consensus.cc:2258] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:09:52.690618 32586 ts_tablet_manager.cc:1905] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:52.690709 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 32136
I20260928 23:09:52.690797 32587 ts_tablet_manager.cc:1905] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:09:52.692766 32586 ts_tablet_manager.cc:1918] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:52.692834 32586 log.cc:1196] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/879310b172884df3bf64c9655253ec4b
I20260928 23:09:52.693121 32586 ts_tablet_manager.cc:1939] T 879310b172884df3bf64c9655253ec4b P 33ee0603ce314f0786172e762c482217: Deleting consensus metadata
I20260928 23:09:52.693512 32085 catalog_manager.cc:4841] TS 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): tablet 879310b172884df3bf64c9655253ec4b (table test-table [id=64a1ac37cf7a4f468fbedaa427ec54bf]) successfully deleted
I20260928 23:09:52.693635 32587 ts_tablet_manager.cc:1918] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:09:52.693707 32587 log.cc:1196] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/879310b172884df3bf64c9655253ec4b
I20260928 23:09:52.693992 32587 ts_tablet_manager.cc:1939] T 879310b172884df3bf64c9655253ec4b P 6751b5727f54403da9d44a8cc4e52ca7: Deleting consensus metadata
I20260928 23:09:52.694420 32084 catalog_manager.cc:4841] TS 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): tablet 879310b172884df3bf64c9655253ec4b (table test-table [id=64a1ac37cf7a4f468fbedaa427ec54bf]) successfully deleted
W20260928 23:09:52.696717 32083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:44051 (error 108)
I20260928 23:09:52.697093 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 32291
I20260928 23:09:52.697144 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:09:52.697036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:44051 (local address 127.0.0.1:53706)
0928 23:09:52.697075 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 23:09:52.697107 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260928 23:09:52.698340 32083 catalog_manager.cc:4584] TS d3044f9c64c64e3a8aaa147b44c83b0c (127.15.212.65:44051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 879310b172884df3bf64c9655253ec4b: Network error: Client connection negotiation failed: client connection to 127.15.212.65:44051: connect: Connection refused (error 111)
I20260928 23:09:52.701711 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 32427
I20260928 23:09:52.706504 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 32065
I20260928 23:10:12.711746 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--webserver_interface=127.15.212.126
--webserver_port=37457
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:32799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:12.791468 32589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:12.791666 32589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:12.791690 32589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:12.792891 32589 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 23:10:12.792928 32589 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 23:10:12.792941 32589 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 23:10:12.792953 32589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:12.793011 32589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:12.793030 32589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:12.793052 32589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:12.793078 32589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:12.794574 32589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:32799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=37457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:12.794799 32589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:12.795029 32589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:12.800274 32594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:12.800318 32589 server_base.cc:1034] running on GCE node
W20260928 23:10:12.800283 32597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:12.800285 32595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:12.800696 32589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:12.801046 32589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:12.802204 32589 hybrid_clock.cc:648] HybridClock initialized: now 1790637012802194 us; error 44 us; skew 500 ppm
Sep 28 23:10:12 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637012, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:12.804638 32589 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:12.805017 32589 webserver.cc:460] Webserver started at http://127.15.212.126:37457/ using document root <none> and password file <none>
I20260928 23:10:12.805222 32589 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:12.805265 32589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:12.806568 32589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:12.807204 32604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:12.807397 32589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:12.807468 32589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "6046e2d01d204bb59888504e00ea65ba"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "8a173a666488bdcd430a4fb22f419b56"
server_key_iv: "7db8b8b288bbe49d5a7c7c506f900e74"
server_key_version: "encryptionkey@0"
I20260928 23:10:12.807850 32589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:12.817587 32589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:12.819829 32589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:12.819939 32589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:12.824090 32589 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:32799
I20260928 23:10:12.824174 32656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:32799 every 8 connection(s)
I20260928 23:10:12.824572 32589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:12.826045 32657 sys_catalog.cc:263] Verifying existing consensus state
I20260928 23:10:12.826422 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32589
I20260928 23:10:12.826682 32657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap starting.
I20260928 23:10:12.826718 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:44051
--local_ip_for_outbound_sockets=127.15.212.65
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=36043
--webserver_interface=127.15.212.65
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:10:12.835268 32657 log.cc:824] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:12.837582 32657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:10:12.837900 32657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Bootstrap complete.
I20260928 23:10:12.840209 32657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:10:12.840478 32657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Initialized, Role: FOLLOWER
I20260928 23:10:12.840682 32657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:10:12.840747 32657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:12.840811 32657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:12.840852 32657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 2 FOLLOWER]: Advancing to term 3
I20260928 23:10:12.841863 32657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:10:12.841984 32657 leader_election.cc:302] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6046e2d01d204bb59888504e00ea65ba; no voters:
I20260928 23:10:12.842149 32657 leader_election.cc:288] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 23:10:12.842204 32661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 3 FOLLOWER]: Leader election won for term 3
I20260928 23:10:12.842329 32661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [term 3 LEADER]: Becoming Leader. State: Replica: 6046e2d01d204bb59888504e00ea65ba, State: Running, Role: LEADER
I20260928 23:10:12.842418 32657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:12.842418 32661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } }
I20260928 23:10:12.842723 32662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6046e2d01d204bb59888504e00ea65ba. Latest consensus state: current_term: 3 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:10:12.842890 32662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:12.842718 32663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6046e2d01d204bb59888504e00ea65ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6046e2d01d204bb59888504e00ea65ba" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32799 } } }
I20260928 23:10:12.843178 32663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:12.843314 32670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:12.844060 32670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:12.844159 32670 catalog_manager.cc:1259] Loaded cluster ID: 90f66fa3b0eb4697aa59edfef23d3064
I20260928 23:10:12.844189 32670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:12.845481 32670 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:12.845664 32670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Loaded TSK: 0
I20260928 23:10:12.845717 32670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6046e2d01d204bb59888504e00ea65ba: Loaded TSK: 1
F20260928 23:10:12.845928 32670 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf996f49ad google::LogMessage::Fail() at ??:0
@ 0x7fdf996f788a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf996f44de google::LogMessage::Flush() at ??:0
@ 0x7fdf996f5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdf9be61104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdf9be5ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fdfa164e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fdfa164e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fdfa164e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fdfa1614c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdfa161ead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdf9b5c20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdf9b5b7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdf9c41b6db start_thread at ??:0
@ 0x7fdf981f261f clone at ??:0
W20260928 23:10:12.929240 32659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:12.929443 32659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:12.929467 32659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:12.930619 32659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:12.930660 32659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:12.930686 32659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:12.932129 32659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:44051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=36043
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:12.932370 32659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:12.932679 32659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:10:12.938045 32659 server_base.cc:1034] running on GCE node
W20260928 23:10:12.938174 32698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:12.938171 32701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:12.938102 32699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:12.938716 32659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:12.939004 32659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:12.940166 32659 hybrid_clock.cc:648] HybridClock initialized: now 1790637012940152 us; error 74 us; skew 500 ppm
Sep 28 23:10:12 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637012, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:12.942703 32659 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:10:12.943173 32659 webserver.cc:460] Webserver started at http://127.15.212.65:36043/ using document root <none> and password file <none>
I20260928 23:10:12.943406 32659 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:12.943477 32659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:12.944896 32659 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:12.945628 32708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:12.945885 32659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:12.945986 32659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d3044f9c64c64e3a8aaa147b44c83b0c"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d5dd92f1bf8c0d5959507e11e7edc334"
server_key_iv: "77296a8942ff3e442fb52c6422a0b43a"
server_key_version: "encryptionkey@0"
I20260928 23:10:12.946449 32659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:12.992623 32659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:12.995185 32659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:12.995352 32659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:12.995635 32659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:12.996179 32715 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 23:10:12.997751 32659 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 23:10:12.997813 32659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 23:10:12.997848 32659 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 23:10:12.998740 32659 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 23:10:12.998786 32659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 23:10:12.998862 32715 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap starting.
I20260928 23:10:13.006162 32659 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:44051
I20260928 23:10:13.006253 354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:44051 every 8 connection(s)
I20260928 23:10:13.006601 32659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:13.009158 32715 log.cc:824] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:13.011272 32715 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 23:10:13.011554 32715 tablet_bootstrap.cc:492] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Bootstrap complete.
I20260928 23:10:13.012514 32715 ts_tablet_manager.cc:1397] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.006s
I20260928 23:10:13.012567 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 32659
I20260928 23:10:13.012907 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:34501
--local_ip_for_outbound_sockets=127.15.212.66
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=34735
--webserver_interface=127.15.212.66
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:10:13.013679 32715 raft_consensus.cc:348] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:13.013921 32715 raft_consensus.cc:729] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3044f9c64c64e3a8aaa147b44c83b0c, State: Initialized, Role: FOLLOWER
I20260928 23:10:13.014091 32715 consensus_queue.cc:260] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:13.014371 32715 ts_tablet_manager.cc:1428] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260928 23:10:13.057890 357 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:10:13.007327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:60637)
0928 23:10:13.007558 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.007623 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:13.008352 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:13.057686 (+ 49334us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
W20260928 23:10:13.058157 355 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:32799 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:32799: Client connection negotiation failed: client connection to 127.15.212.126:32799: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:10:13.058559 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.058427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:60593)
0928 23:10:13.058482 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.058534 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:10:13.059005 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.058914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:56271)
0928 23:10:13.058957 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.058985 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 23:10:13.059115 355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 23:10:13.095754 359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:13.095957 359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:13.095980 359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:13.097188 359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:13.097230 359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:13.097278 359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:13.098716 359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:34501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=34735
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:13.098950 359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:13.099179 359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:13.104054 368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:13.104140 359 server_base.cc:1034] running on GCE node
W20260928 23:10:13.104135 365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:13.104055 366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:13.104545 359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:13.104800 359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:13.105967 359 hybrid_clock.cc:648] HybridClock initialized: now 1790637013105948 us; error 31 us; skew 500 ppm
Sep 28 23:10:13 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637013, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:13.108392 359 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:10:13.108855 359 webserver.cc:460] Webserver started at http://127.15.212.66:34735/ using document root <none> and password file <none>
I20260928 23:10:13.109097 359 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:13.109167 359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:13.110437 359 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:10:13.110998 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.111152 359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:13.111249 359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "6751b5727f54403da9d44a8cc4e52ca7"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "d7c4bb53310b87d2fcaa565e806d9e4b"
server_key_iv: "c9d34db19278c1e6967dbd16db9cf78a"
server_key_version: "encryptionkey@0"
I20260928 23:10:13.111593 359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:13.122466 359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:13.124279 359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:13.124473 359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:13.124702 359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:13.125006 359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:13.125061 359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.125100 359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:13.125128 359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.130746 359 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:34501
I20260928 23:10:13.130816 488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:34501 every 8 connection(s)
I20260928 23:10:13.131142 359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:10:13.132109 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.131756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:56011)
0928 23:10:13.131900 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.131975 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
W20260928 23:10:13.132619 489 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:32799 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:32799: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
I20260928 23:10:13.132890 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.132802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:44397)
0928 23:10:13.132848 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.132874 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:10:13.133152 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.133080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:40331)
0928 23:10:13.133118 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.133138 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 23:10:13.133255 489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 23:10:13.138222 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 359
I20260928 23:10:13.138499 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:41823
--local_ip_for_outbound_sockets=127.15.212.67
--tserver_master_addrs=127.15.212.126:32799
--webserver_port=41629
--webserver_interface=127.15.212.67
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:13.218225 492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:13.218420 492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:13.218442 492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:13.219594 492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:13.219635 492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:13.219687 492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:13.221103 492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:41823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=41629
--tserver_master_addrs=127.15.212.126:32799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:13.221334 492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:13.221560 492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:13.226385 497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:13.226459 498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:13.226501 500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:13.226555 492 server_base.cc:1034] running on GCE node
I20260928 23:10:13.226854 492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:13.227082 492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:13.228247 492 hybrid_clock.cc:648] HybridClock initialized: now 1790637013228236 us; error 27 us; skew 500 ppm
Sep 28 23:10:13 dist-test-slave-x2t8 krb5kdc[31517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637013, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:13.230718 492 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:10:13.231189 492 webserver.cc:460] Webserver started at http://127.15.212.67:41629/ using document root <none> and password file <none>
I20260928 23:10:13.231406 492 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:13.231457 492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:13.232829 492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:10:13.233397 507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.233577 492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:13.233664 492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "33ee0603ce314f0786172e762c482217"
format_stamp: "Formatted at 2026-09-28 23:09:31 on dist-test-slave-x2t8"
server_key: "aafae1c204d27c2dacbc5fb5e70917eb"
server_key_iv: "d0634545f1e365c71b9b2afbecdf5d01"
server_key_version: "encryptionkey@0"
I20260928 23:10:13.233986 492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:13.261466 492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:13.263374 492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:13.263569 492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:13.263809 492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:13.264120 492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:13.264173 492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.264213 492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:13.264241 492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:13.270457 492 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:41823
I20260928 23:10:13.270498 620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:41823 every 8 connection(s)
I20260928 23:10:13.270819 492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:10:13.271888 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.271412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:46535)
0928 23:10:13.271643 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.271749 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
W20260928 23:10:13.272285 621 heartbeater.cc:644] Failed to heartbeat to 127.15.212.126:32799 (0 consecutive failures): Network error: Failed to ping master at 127.15.212.126:32799: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
I20260928 23:10:13.272608 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.272516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:39787)
0928 23:10:13.272559 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.272589 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 23:10:13.272883 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:13.272799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:47751)
0928 23:10:13.272840 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:13.272862 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 23:10:13.272979 621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 23:10:13.273371 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 492
I20260928 23:10:14.059619 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.059392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:34589)
0928 23:10:14.059512 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:14.059545 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260928 23:10:14.133721 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.133507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:39079)
0928 23:10:14.133641 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:14.133676 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260928 23:10:14.241501 627 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:14.241613 627 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:14.241999 627 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
Sep 28 23:10:14 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637012, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.66@KRBTEST.COM
Sep 28 23:10:14 dist-test-slave-x2t8 krb5kdc[31517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637012, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.67@KRBTEST.COM
I20260928 23:10:14.253728 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.242286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:56711 (local address 127.15.212.67:41823)
0928 23:10:14.242434 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:14.242441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:14.242462 (+ 21us) server_negotiation.cc:407] Connection header received
0928 23:10:14.243032 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:14.243035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:14.243097 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:14.243247 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:14.244191 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.245572 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247108 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.247529 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247632 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:14.250404 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:14.250408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:14.250408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:14.250420 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:14.252169 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252532 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:14.252534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:14.252535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:14.252628 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252816 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:14.252817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:14.252817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:14.253070 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:14.253141 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:14.253491 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 23:10:14.253623 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260928 23:10:14.253728 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.242259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48661 (local address 127.15.212.66:34501)
0928 23:10:14.242389 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:14.242401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:14.242437 (+ 36us) server_negotiation.cc:407] Connection header received
0928 23:10:14.243030 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:14.243035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:14.243097 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:14.243248 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:14.244191 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.245572 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247091 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.247539 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247656 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:14.249953 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:14.249957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:14.249957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:14.249967 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:14.251911 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252325 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:14.252328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:14.252329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:14.252410 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252732 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:14.252733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:14.252733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:14.253042 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:14.253132 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:14.253509 (+ 377us) server_negotiation.cc:299] Negotiation successful
0928 23:10:14.253621 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260928 23:10:14.253729 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.242149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:34501 (local address 127.15.212.65:48661)
0928 23:10:14.242208 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:14.242218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:14.242273 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:14.243308 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:14.243311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:14.243764 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:14.244071 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:14.244078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.245680 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.245681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:14.246986 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:14.246988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.247637 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:14.247731 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:14.248386 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:14.248390 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:14.249823 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:14.252005 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:14.252014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:14.252247 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:14.252534 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:14.252535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:14.252645 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:14.253131 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:14.253139 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:14.253294 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 23:10:14.253420 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 23:10:14.253614 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":95}
I20260928 23:10:14.253785 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.242167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:41823 (local address 127.15.212.65:56711)
0928 23:10:14.242307 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:14.242318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:14.242373 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:14.243308 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:14.243311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:14.243773 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:14.244071 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:14.244078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.245680 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.245681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:14.246986 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:14.246988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:14.247617 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:14.247618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:14.247733 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:14.248300 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:14.248326 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:14.250289 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:14.252246 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:14.252247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:14.252413 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:14.252690 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:14.252691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:14.252691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:14.252749 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:14.253132 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:14.253139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:14.253294 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 23:10:14.253420 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 23:10:14.253666 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":238,"thread_start_us":49,"threads_started":1}
W20260928 23:10:14.254562 32711 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
W20260928 23:10:14.255020 32712 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
I20260928 23:10:14.255112 32712 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3044f9c64c64e3a8aaa147b44c83b0c; no voters: 33ee0603ce314f0786172e762c482217, 6751b5727f54403da9d44a8cc4e52ca7
I20260928 23:10:14.255302 627 raft_consensus.cc:2726] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:10:14.273408 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:14.273249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:55963)
0928 23:10:14.273346 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:14.273381 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":48,"threads_started":1}
I20260928 23:10:15.060315 632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:15.060053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:45235)
0928 23:10:15.060213 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:15.060260 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260928 23:10:15.134277 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:15.134114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:56019)
0928 23:10:15.134217 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:15.134243 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":52,"threads_started":1}
I20260928 23:10:15.273978 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:15.273758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:37675)
0928 23:10:15.273894 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:15.273929 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":56,"threads_started":1}
I20260928 23:10:16.061009 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:16.060794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:39405)
0928 23:10:16.060913 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:16.060946 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260928 23:10:16.134866 636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:16.134667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:46135)
0928 23:10:16.134796 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:16.134830 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260928 23:10:16.240985 637 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:16.241072 637 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:16.241376 637 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
W20260928 23:10:16.241741 32712 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
W20260928 23:10:16.241824 32711 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
I20260928 23:10:16.241933 32711 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3044f9c64c64e3a8aaa147b44c83b0c; no voters: 33ee0603ce314f0786172e762c482217, 6751b5727f54403da9d44a8cc4e52ca7
I20260928 23:10:16.242034 637 raft_consensus.cc:2726] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:10:16.274580 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:16.274411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:35385)
0928 23:10:16.274516 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:16.274544 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260928 23:10:17.061719 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:17.061512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:46049)
0928 23:10:17.061638 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:17.061674 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260928 23:10:17.135491 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:17.135312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:44439)
0928 23:10:17.135422 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:17.135455 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260928 23:10:17.275122 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:17.274937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:39597)
0928 23:10:17.275051 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:17.275087 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260928 23:10:18.062400 642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:18.062214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:59901)
0928 23:10:18.062323 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:18.062356 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260928 23:10:18.136086 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:18.135904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:55579)
0928 23:10:18.136020 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:18.136051 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260928 23:10:18.275683 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:18.275499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:42451)
0928 23:10:18.275618 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:18.275647 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260928 23:10:18.624658 645 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:18.624753 645 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:18.624922 645 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
W20260928 23:10:18.625308 32712 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
W20260928 23:10:18.625393 32711 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
I20260928 23:10:18.625437 32711 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3044f9c64c64e3a8aaa147b44c83b0c; no voters: 33ee0603ce314f0786172e762c482217, 6751b5727f54403da9d44a8cc4e52ca7
I20260928 23:10:18.625542 645 raft_consensus.cc:2726] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:10:19.063066 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:19.062863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:55971)
0928 23:10:19.062985 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:19.063020 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260928 23:10:19.136781 647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:19.136557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:59235)
0928 23:10:19.136710 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:19.136743 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260928 23:10:19.276269 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:19.276078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:43001)
0928 23:10:19.276202 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:19.276232 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260928 23:10:20.063735 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:20.063506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:37053)
0928 23:10:20.063656 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:20.063691 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260928 23:10:20.137399 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:20.137181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:40081)
0928 23:10:20.137322 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:20.137363 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260928 23:10:20.276896 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:20.276672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:57695)
0928 23:10:20.276805 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:20.276839 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260928 23:10:21.064327 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:21.064140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:35563)
0928 23:10:21.064253 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:21.064282 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260928 23:10:21.137964 653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:21.137781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:53075)
0928 23:10:21.137893 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:21.137928 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260928 23:10:21.277590 654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:21.277400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:45031)
0928 23:10:21.277516 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:21.277552 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260928 23:10:22.065129 655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:22.064908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:50601)
0928 23:10:22.065042 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:22.065082 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260928 23:10:22.138581 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:22.138373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:43515)
0928 23:10:22.138504 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:22.138545 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260928 23:10:22.278169 657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:22.277990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:40585)
0928 23:10:22.278100 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:22.278130 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260928 23:10:22.311236 658 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:22.311323 658 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:22.311527 658 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
W20260928 23:10:22.311898 32711 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
W20260928 23:10:22.312000 32712 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
I20260928 23:10:22.312053 32712 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3044f9c64c64e3a8aaa147b44c83b0c; no voters: 33ee0603ce314f0786172e762c482217, 6751b5727f54403da9d44a8cc4e52ca7
I20260928 23:10:22.312134 658 raft_consensus.cc:2726] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:10:23.065798 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:23.065592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:51693)
0928 23:10:23.065709 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:23.065741 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260928 23:10:23.139163 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:23.138960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:49413)
0928 23:10:23.139094 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:23.139123 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260928 23:10:23.278791 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:23.278595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:34719)
0928 23:10:23.278724 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:23.278752 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260928 23:10:24.066507 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:24.066304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:33075)
0928 23:10:24.066423 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:24.066459 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260928 23:10:24.139707 663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:24.139543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:50697)
0928 23:10:24.139644 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:24.139668 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260928 23:10:24.279543 664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:24.279337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:57889)
0928 23:10:24.279465 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:24.279501 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260928 23:10:25.067176 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:25.066960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:45455)
0928 23:10:25.067096 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:25.067132 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260928 23:10:25.140329 666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:25.140135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:39075)
0928 23:10:25.140254 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:25.140288 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260928 23:10:25.280268 667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:25.280072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:58427)
0928 23:10:25.280192 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:25.280227 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260928 23:10:25.552359 668 raft_consensus.cc:482] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:25.552479 668 raft_consensus.cc:504] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6751b5727f54403da9d44a8cc4e52ca7" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 34501 } } peers { permanent_uuid: "33ee0603ce314f0786172e762c482217" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 41823 } } peers { permanent_uuid: "d3044f9c64c64e3a8aaa147b44c83b0c" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 44051 } }
I20260928 23:10:25.552649 668 leader_election.cc:288] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501), 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823)
W20260928 23:10:25.552964 32711 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6751b5727f54403da9d44a8cc4e52ca7 (127.15.212.66:34501): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
W20260928 23:10:25.553068 32712 leader_election.cc:341] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33ee0603ce314f0786172e762c482217 (127.15.212.67:41823): Not found: Tablet not found: 879310b172884df3bf64c9655253ec4b
I20260928 23:10:25.553150 32712 leader_election.cc:302] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3044f9c64c64e3a8aaa147b44c83b0c; no voters: 33ee0603ce314f0786172e762c482217, 6751b5727f54403da9d44a8cc4e52ca7
I20260928 23:10:25.553247 668 raft_consensus.cc:2726] T 879310b172884df3bf64c9655253ec4b P d3044f9c64c64e3a8aaa147b44c83b0c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 23:10:26.067842 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:26.067642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:45517)
0928 23:10:26.067759 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:26.067796 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260928 23:10:26.140985 670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:26.140797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:44559)
0928 23:10:26.140911 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:26.140944 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260928 23:10:26.280885 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:26.280711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:39999)
0928 23:10:26.280820 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:26.280850 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260928 23:10:27.068531 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:27.068265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:57415)
0928 23:10:27.068395 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:27.068427 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260928 23:10:27.141613 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:27.141419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:40559)
0928 23:10:27.141541 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:27.141575 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260928 23:10:27.281543 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:27.281300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:47215)
0928 23:10:27.281458 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:27.281495 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260928 23:10:28.069224 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:28.069026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.65:37291)
0928 23:10:28.069145 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:28.069179 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260928 23:10:28.142267 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:28.142069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.66:46519)
0928 23:10:28.142192 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:28.142223 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260928 23:10:28.278334 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 32659
I20260928 23:10:28.282263 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:28.282077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32799 (local address 127.15.212.67:45171)
0928 23:10:28.282195 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:28.282227 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.126:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260928 23:10:28.283432 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 359
I20260928 23:10:28.287531 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 492
2026-09-28T23:10:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58385 ms)
[----------] 30 tests from SecurityITest (229533 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:28 dist-test-slave-x2t8 krb5kdc[680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:28 dist-test-slave-x2t8 krb5kdc[680](info): set up 2 sockets
Sep 28 23:10:28 dist-test-slave-x2t8 krb5kdc[680](info): commencing operation
krb5kdc: starting...
W20260928 23:10:29.332108 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:29.448365 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44323
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:44323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:29.529592 693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:29.529853 693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:29.529894 693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:29.531225 693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:29.531301 693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:29.531329 693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:29.531365 693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:29.531401 693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:29.533006 693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:44323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:29.533301 693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:29.533555 693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:29.539275 701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:29.539297 693 server_base.cc:1034] running on GCE node
W20260928 23:10:29.539242 698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:29.539249 699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:29.539819 693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:29.540148 693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:29.541332 693 hybrid_clock.cc:648] HybridClock initialized: now 1790637029541308 us; error 34 us; skew 500 ppm
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:29.543897 693 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:29.544396 693 webserver.cc:460] Webserver started at http://127.15.212.126:41905/ using document root <none> and password file <none>
I20260928 23:10:29.544661 693 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:29.544735 693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:29.544894 693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:29.546015 693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "3e32643711ba4851b4ff120c8dd8ecda"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "917fdd82df22faddf6662173ee5cd133"
server_key_iv: "6f8bb54d4b95a42721ab6311fb58b411"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.546427 693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "3e32643711ba4851b4ff120c8dd8ecda"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "917fdd82df22faddf6662173ee5cd133"
server_key_iv: "6f8bb54d4b95a42721ab6311fb58b411"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.547822 693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:29.548624 708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.548805 693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:29.548941 693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "3e32643711ba4851b4ff120c8dd8ecda"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "917fdd82df22faddf6662173ee5cd133"
server_key_iv: "6f8bb54d4b95a42721ab6311fb58b411"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.549029 693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:29.566794 693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:29.569348 693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:29.569537 693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:29.573756 693 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:44323
I20260928 23:10:29.573793 760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:44323 every 8 connection(s)
I20260928 23:10:29.574159 693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:29.575026 761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:29.577237 761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: Bootstrap starting.
I20260928 23:10:29.577786 761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:29.578013 761 log.cc:824] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:29.578680 761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: No bootstrap required, opened a new log
I20260928 23:10:29.580183 761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } }
I20260928 23:10:29.580314 761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:29.580336 761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e32643711ba4851b4ff120c8dd8ecda, State: Initialized, Role: FOLLOWER
I20260928 23:10:29.580495 761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } }
I20260928 23:10:29.580586 761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:29.580610 761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:29.580638 761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:29.581099 761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } }
I20260928 23:10:29.581178 761 leader_election.cc:302] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e32643711ba4851b4ff120c8dd8ecda; no voters:
I20260928 23:10:29.581295 761 leader_election.cc:288] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:29.581350 764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:29.581539 761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:29.581555 764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [term 1 LEADER]: Becoming Leader. State: Replica: 3e32643711ba4851b4ff120c8dd8ecda, State: Running, Role: LEADER
I20260928 23:10:29.581733 764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } }
I20260928 23:10:29.582134 766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e32643711ba4851b4ff120c8dd8ecda. Latest consensus state: current_term: 1 leader_uuid: "3e32643711ba4851b4ff120c8dd8ecda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } } }
I20260928 23:10:29.582237 766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:29.582255 765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e32643711ba4851b4ff120c8dd8ecda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e32643711ba4851b4ff120c8dd8ecda" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44323 } } }
I20260928 23:10:29.582330 765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:29.582633 773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:29.583163 773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:29.583353 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 693
I20260928 23:10:29.583461 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:29.583659 16209 external_mini_cluster.cc:1428] Setting key bb55f7a8f508d0f7dc4c0b59c476fb19
I20260928 23:10:29.588856 773 catalog_manager.cc:1347] Generated new cluster ID: 472a9b434a9e4f1181ae679510dacfc9
I20260928 23:10:29.588912 773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:29.594439 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:29.584120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.15.212.126:44323)
0928 23:10:29.584372 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:29.584385 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:29.584420 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:10:29.584559 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:29.584565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:29.584606 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:29.584714 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:29.585499 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.586361 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.587479 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.587903 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.587985 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:29.590785 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:29.590789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:29.590789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:29.590801 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:29.592827 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.593216 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.593219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.593220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.593343 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.593546 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.593546 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.593547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.593842 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:29.593999 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:29.594079 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 23:10:29.594213 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":69,"threads_started":1}
I20260928 23:10:29.609843 773 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:29.610353 773 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:29.613329 773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: Generated new TSK 0
I20260928 23:10:29.613528 773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:10:29.642946 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:29.742981 787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:29.743181 787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:29.743204 787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:29.744235 787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:29.744268 787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:29.744294 787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:29.745639 787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:29.745814 787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:29.745980 787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:29.750816 795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:29.750903 793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:29.751076 787 server_base.cc:1034] running on GCE node
W20260928 23:10:29.750911 792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:29.751372 787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:29.751601 787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:29.752795 787 hybrid_clock.cc:648] HybridClock initialized: now 1790637029752772 us; error 34 us; skew 500 ppm
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:29.755214 787 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:10:29.755658 787 webserver.cc:460] Webserver started at http://127.15.212.65:43321/ using document root <none> and password file <none>
I20260928 23:10:29.755872 787 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:29.755936 787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:29.756064 787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:29.757149 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "d31e3e5a8114461ebdbf4a20675ca308"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "3024e9ea0b7d6d572d85ae6c7739d7bf"
server_key_iv: "fba1681ceec8a2aca68840cb00bd6ea3"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.757519 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "d31e3e5a8114461ebdbf4a20675ca308"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "3024e9ea0b7d6d572d85ae6c7739d7bf"
server_key_iv: "fba1681ceec8a2aca68840cb00bd6ea3"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.758790 787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:10:29.759488 802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.759690 787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:29.759797 787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d31e3e5a8114461ebdbf4a20675ca308"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "3024e9ea0b7d6d572d85ae6c7739d7bf"
server_key_iv: "fba1681ceec8a2aca68840cb00bd6ea3"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.759888 787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:29.774967 787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:29.777354 787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:29.777521 787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:29.777767 787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:29.778085 787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:29.778141 787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.778182 787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:29.778210 787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.784349 787 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:45155
I20260928 23:10:29.784399 915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:45155 every 8 connection(s)
I20260928 23:10:29.784727 787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:29.787942 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 787
I20260928 23:10:29.788035 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:29.788256 16209 external_mini_cluster.cc:1428] Setting key 1a0ec3c02157477d07af84465d13fd95
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:29.795033 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:29.785414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:59149 (local address 127.15.212.126:44323)
0928 23:10:29.785457 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:29.785459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:29.785642 (+ 183us) server_negotiation.cc:407] Connection header received
0928 23:10:29.786120 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:29.786121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:29.786133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:29.786183 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:29.787052 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.787550 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.788790 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.789071 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.789132 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:29.791896 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:29.791898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:29.791898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:29.791906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:29.793532 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.794046 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.794046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.794047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.794098 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.794412 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.794412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.794412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.794511 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:29.794563 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:29.794900 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 23:10:29.794970 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:10:29.795357 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:29.785397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44323 (local address 127.15.212.65:59149)
0928 23:10:29.785543 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:29.785577 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:29.786034 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:29.786232 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:29.786235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:29.786774 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:29.786971 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:29.786975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.787630 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.787631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:29.788713 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:29.788715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.789176 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.789177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:29.789319 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:29.789835 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:29.789845 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:29.791790 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:29.793622 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:29.793631 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:29.793633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:29.793968 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:29.794167 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:29.794168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:29.794169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:29.794320 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:29.794578 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:29.794586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:29.794756 (+ 170us) client_negotiation.cc:769] Sending connection context
0928 23:10:29.794879 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 23:10:29.795047 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":90,"threads_started":1}
I20260928 23:10:29.795928 916 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44323
I20260928 23:10:29.796026 916 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:29.796221 916 heartbeater.cc:511] Master 127.15.212.126:44323 requested a full tablet report, sending...
I20260928 23:10:29.796836 725 ts_manager.cc:194] Registered new tserver with Master: d31e3e5a8114461ebdbf4a20675ca308 (127.15.212.65:45155)
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
I20260928 23:10:29.797531 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:59149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:10:29.814990 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:29.893690 921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:29.893924 921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:29.893959 921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:29.895109 921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:29.895175 921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:29.895218 921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:29.896642 921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:29.896914 921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:29.897141 921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:29.901939 929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:29.901949 926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:29.901943 927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:29.901973 921 server_base.cc:1034] running on GCE node
I20260928 23:10:29.902393 921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:29.902622 921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:29.903779 921 hybrid_clock.cc:648] HybridClock initialized: now 1790637029903756 us; error 34 us; skew 500 ppm
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:29.906154 921 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:10:29.906631 921 webserver.cc:460] Webserver started at http://127.15.212.66:38271/ using document root <none> and password file <none>
I20260928 23:10:29.906852 921 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:29.906917 921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:29.907068 921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:29.908057 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "de261b4509ab4c0099ca8e00969ea588"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "6b504e83947d8b86ec2a85854ad84473"
server_key_iv: "b0e254cf7ffcdf2a5640bb6dee7dbccb"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.908412 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "de261b4509ab4c0099ca8e00969ea588"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "6b504e83947d8b86ec2a85854ad84473"
server_key_iv: "b0e254cf7ffcdf2a5640bb6dee7dbccb"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.909701 921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:29.910400 936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.910565 921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:29.910658 921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "de261b4509ab4c0099ca8e00969ea588"
format_stamp: "Formatted at 2026-09-28 23:10:29 on dist-test-slave-x2t8"
server_key: "6b504e83947d8b86ec2a85854ad84473"
server_key_iv: "b0e254cf7ffcdf2a5640bb6dee7dbccb"
server_key_version: "encryptionkey@0"
I20260928 23:10:29.910737 921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:29.929188 921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:29.931084 921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:29.931248 921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:29.931466 921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:29.931787 921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:29.931842 921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.931881 921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:29.931907 921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:29.938192 921 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:33245
I20260928 23:10:29.938241 1049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:33245 every 8 connection(s)
I20260928 23:10:29.938539 921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:10:29.939581 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 921
I20260928 23:10:29.939725 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:29.939913 16209 external_mini_cluster.cc:1428] Setting key 417a64a9be57a1acc600afaf60f26e59
Sep 28 23:10:29 dist-test-slave-x2t8 krb5kdc[680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637029, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
I20260928 23:10:29.948967 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:29.939167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:48821 (local address 127.15.212.126:44323)
0928 23:10:29.939247 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:29.939249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:29.939440 (+ 191us) server_negotiation.cc:407] Connection header received
0928 23:10:29.940061 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:29.940063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:29.940076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:29.940123 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:29.941122 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.941902 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.943307 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.943649 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.943723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:29.945906 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:29.945908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:29.945908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:29.945916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:29.947623 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.948026 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.948028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.948028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.948088 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:29.948323 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:29.948324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:29.948324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:29.948495 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:29.948582 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:29.948813 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 23:10:29.948888 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 23:10:29.949148 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:29.939122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44323 (local address 127.15.212.66:48821)
0928 23:10:29.939272 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:29.939306 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:29.939932 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:29.940209 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:29.940213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:29.940820 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:29.941009 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:29.941014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.942007 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.942008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:29.943219 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:29.943222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:29.943816 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:29.943817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:29.943925 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:29.944498 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:29.944508 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:29.945817 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:29.947718 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:29.947724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:29.947726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:29.947944 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:29.948157 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:29.948157 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:29.948158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:29.948251 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:29.948571 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:29.948576 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:29.948687 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 23:10:29.948773 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 23:10:29.948918 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260928 23:10:29.949607 1050 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44323
I20260928 23:10:29.949687 1050 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:29.949815 1050 heartbeater.cc:511] Master 127.15.212.126:44323 requested a full tablet report, sending...
I20260928 23:10:29.950143 725 ts_manager.cc:194] Registered new tserver with Master: de261b4509ab4c0099ca8e00969ea588 (127.15.212.66:33245)
I20260928 23:10:29.950598 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:48821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:10:29.965564 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:30.046178 1055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:30.046386 1055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:30.046419 1055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:30.047508 1055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:30.047572 1055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:30.047612 1055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:30.049124 1055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:44323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:30.049371 1055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:30.049578 1055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:30.054352 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.054399 1061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.054575 1060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.054561 1055 server_base.cc:1034] running on GCE node
I20260928 23:10:30.054790 1055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:30.054977 1055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:30.056110 1055 hybrid_clock.cc:648] HybridClock initialized: now 1790637030056107 us; error 30 us; skew 500 ppm
Sep 28 23:10:30 dist-test-slave-x2t8 krb5kdc[680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637030, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:30.058583 1055 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:10:30.059032 1055 webserver.cc:460] Webserver started at http://127.15.212.67:43475/ using document root <none> and password file <none>
I20260928 23:10:30.059262 1055 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:30.059305 1055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:30.059469 1055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:30.060493 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "a1cad91a86b641d8968af2c39711c80f"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "3d3882185e0a6a5bc18902284d26bf4a"
server_key_iv: "0ca4f2ddeeca6f86af7ed57646a1e60d"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.060846 1055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "a1cad91a86b641d8968af2c39711c80f"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "3d3882185e0a6a5bc18902284d26bf4a"
server_key_iv: "0ca4f2ddeeca6f86af7ed57646a1e60d"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.062109 1055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:30.062990 1070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.063192 1055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:30.063264 1055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "a1cad91a86b641d8968af2c39711c80f"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "3d3882185e0a6a5bc18902284d26bf4a"
server_key_iv: "0ca4f2ddeeca6f86af7ed57646a1e60d"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.063352 1055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:30.090929 1055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:30.092854 1055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:30.093009 1055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:30.093241 1055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:30.093552 1055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:30.093586 1055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.093631 1055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:30.093657 1055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.099645 1055 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:43635
I20260928 23:10:30.099686 1183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:43635 every 8 connection(s)
I20260928 23:10:30.100049 1055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:10:30 dist-test-slave-x2t8 krb5kdc[680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637030, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:30.108700 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.100772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48055 (local address 127.15.212.126:44323)
0928 23:10:30.100856 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.100858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.101020 (+ 162us) server_negotiation.cc:407] Connection header received
0928 23:10:30.101501 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.101502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.101513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.101592 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.102444 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.102923 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.103847 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.104034 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.104083 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.106041 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.106043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.106043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:30.106049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.107471 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:30.107830 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:30.107831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:30.107831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:30.107881 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:30.108084 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:30.108085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:30.108085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:30.108181 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.108232 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.108553 (+ 321us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.108637 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 23:10:30.108737 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.100733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:44323 (local address 127.15.212.67:48055)
0928 23:10:30.100885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.100919 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.101396 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.101647 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.101650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.102189 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:30.102367 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.102372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.102997 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.102998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.103776 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.103778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.104101 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.104101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.104199 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.104708 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:30.104715 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:30.105955 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:30.107552 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:30.107557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:30.107559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:30.107763 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:30.107938 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:30.107939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:30.107939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:30.108030 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:30.108231 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:30.108235 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:30.108339 (+ 104us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.108418 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.108588 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260928 23:10:30.109225 1184 heartbeater.cc:347] Connected to a master server at 127.15.212.126:44323
I20260928 23:10:30.109303 1184 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:30.109484 1184 heartbeater.cc:511] Master 127.15.212.126:44323 requested a full tablet report, sending...
I20260928 23:10:30.109885 725 ts_manager.cc:194] Registered new tserver with Master: a1cad91a86b641d8968af2c39711c80f (127.15.212.67:43635)
I20260928 23:10:30.110194 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1055
I20260928 23:10:30.110225 725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:48055
I20260928 23:10:30.110296 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:30.110491 16209 external_mini_cluster.cc:1428] Setting key 1712a83274204071eba32802670c9560
I20260928 23:10:30.111317 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:30.118784 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.112564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.15.212.126:44323)
0928 23:10:30.112601 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.112603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.112726 (+ 123us) server_negotiation.cc:407] Connection header received
0928 23:10:30.112804 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.112805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.112814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.112869 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.113458 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.114036 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.114791 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.115013 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.115101 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.116258 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.116259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.116260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:30.116266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.117627 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:30.117996 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:30.117997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:30.117997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:30.118048 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:30.118260 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:30.118261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:30.118261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:30.118400 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.118510 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.118613 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.118687 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:10:30.120396 725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:30.122001 725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:30.129765 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:43635 (local address 127.0.0.1:39028)
0928 23:10:30.125012 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.125055 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.125131 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.125429 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.125433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.125435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.125524 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.125526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.127180 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.127181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.128518 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.128522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.129485 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.129486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.129561 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.129576 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.129641 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.129692 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":57,"threads_started":1}
I20260928 23:10:30.130276 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.15.212.67:43635)
0928 23:10:30.125054 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.125059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.125220 (+ 161us) server_negotiation.cc:407] Connection header received
0928 23:10:30.125263 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.125266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.125283 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.125351 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.125592 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.126916 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.128614 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.129340 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.129397 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.129454 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.130154 (+ 700us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.130195 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260928 23:10:30.130596 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:45155 (local address 127.0.0.1:46396)
0928 23:10:30.125363 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.125373 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.125436 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.125712 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.125713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.125715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.125757 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.125758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.128222 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.128223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.129344 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.129348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.130348 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.130349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.130415 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.130419 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.130481 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.130520 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":82,"threads_started":1}
I20260928 23:10:30.130869 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46396 (local address 127.15.212.65:45155)
0928 23:10:30.125230 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.125233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.125465 (+ 232us) server_negotiation.cc:407] Connection header received
0928 23:10:30.125535 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.125538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.125554 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.125626 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.125847 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.128108 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.129446 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.130182 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.130257 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.130316 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.130748 (+ 432us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.130792 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":101,"threads_started":1}
I20260928 23:10:30.131201 1118 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90a3a35b41447398a43a6c54d76232 (DEFAULT_TABLE table=test-table [id=bfc35cc705094e88b1360328539debcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.131528 1118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90a3a35b41447398a43a6c54d76232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.131479 850 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90a3a35b41447398a43a6c54d76232 (DEFAULT_TABLE table=test-table [id=bfc35cc705094e88b1360328539debcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.131807 850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90a3a35b41447398a43a6c54d76232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.131955 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:33245 (local address 127.0.0.1:37284)
0928 23:10:30.126945 (+ 2237us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.126956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.127029 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.127510 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.127511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.127513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.127589 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.127591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.129338 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.129339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.130763 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.130766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.131720 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.131721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.131783 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.131787 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.131849 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.131888 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2223,"thread_start_us":66,"threads_started":1}
I20260928 23:10:30.132011 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.124827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.15.212.66:33245)
0928 23:10:30.125145 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.125149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.127071 (+ 1922us) server_negotiation.cc:407] Connection header received
0928 23:10:30.127148 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.127150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.127170 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.127270 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.127690 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.129196 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.130866 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.131565 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.131635 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.131684 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.131879 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.131924 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":86,"threads_started":1}
I20260928 23:10:30.132674 984 tablet_service.cc:1467] Processing CreateTablet for tablet 3a90a3a35b41447398a43a6c54d76232 (DEFAULT_TABLE table=test-table [id=bfc35cc705094e88b1360328539debcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.132972 984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a90a3a35b41447398a43a6c54d76232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.134326 1201 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Bootstrap starting.
I20260928 23:10:30.134956 1201 tablet_bootstrap.cc:654] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.135202 1202 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: Bootstrap starting.
I20260928 23:10:30.135555 1201 log.cc:824] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.135852 1202 tablet_bootstrap.cc:654] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.136009 1204 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Bootstrap starting.
I20260928 23:10:30.136396 1201 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: No bootstrap required, opened a new log
I20260928 23:10:30.136471 1202 log.cc:824] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.136507 1201 ts_tablet_manager.cc:1397] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:30.136685 1204 tablet_bootstrap.cc:654] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.137261 1204 log.cc:824] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.137305 1202 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: No bootstrap required, opened a new log
I20260928 23:10:30.137387 1202 ts_tablet_manager.cc:1397] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:30.138087 1204 tablet_bootstrap.cc:492] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: No bootstrap required, opened a new log
I20260928 23:10:30.138200 1204 ts_tablet_manager.cc:1397] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:30.138432 1201 raft_consensus.cc:348] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.138590 1201 raft_consensus.cc:374] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.138650 1201 raft_consensus.cc:729] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cad91a86b641d8968af2c39711c80f, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.138773 1201 consensus_queue.cc:260] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.139060 1201 ts_tablet_manager.cc:1428] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260928 23:10:30.139202 1184 heartbeater.cc:503] Master 127.15.212.126:44323 was elected leader, sending a full tablet report...
I20260928 23:10:30.139443 1202 raft_consensus.cc:348] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.139616 1202 raft_consensus.cc:374] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.139693 1202 raft_consensus.cc:729] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31e3e5a8114461ebdbf4a20675ca308, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.139807 1202 consensus_queue.cc:260] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.140106 916 heartbeater.cc:503] Master 127.15.212.126:44323 was elected leader, sending a full tablet report...
I20260928 23:10:30.140091 1202 ts_tablet_manager.cc:1428] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 23:10:30.140223 1204 raft_consensus.cc:348] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.140383 1204 raft_consensus.cc:374] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.140488 1204 raft_consensus.cc:729] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de261b4509ab4c0099ca8e00969ea588, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.140606 1204 consensus_queue.cc:260] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.140861 1204 ts_tablet_manager.cc:1428] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 23:10:30.140901 1050 heartbeater.cc:503] Master 127.15.212.126:44323 was elected leader, sending a full tablet report...
I20260928 23:10:30.168269 1208 raft_consensus.cc:482] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:30.168401 1208 raft_consensus.cc:504] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.168797 1208 leader_election.cc:288] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1cad91a86b641d8968af2c39711c80f (127.15.212.67:43635), de261b4509ab4c0099ca8e00969ea588 (127.15.212.66:33245)
I20260928 23:10:30.172605 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.169112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:34369 (local address 127.15.212.66:33245)
0928 23:10:30.169153 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.169155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.169329 (+ 174us) server_negotiation.cc:407] Connection header received
0928 23:10:30.169354 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.169354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.169363 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.169539 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.169672 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.170371 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.171693 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.172267 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.172341 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.172349 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.172510 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.172555 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 23:10:30.172657 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.168940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:43635 (local address 127.15.212.65:50497)
0928 23:10:30.169006 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.169026 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.169078 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.169262 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.169263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.169265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.169344 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.169346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.170379 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.170379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.171599 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.171602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.172353 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.172354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.172429 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.172430 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.172503 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.172565 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":4}
I20260928 23:10:30.172663 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.169064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:33245 (local address 127.15.212.65:34369)
0928 23:10:30.169246 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.169254 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.169304 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.169553 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.169554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.169555 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.169606 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.169607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.170455 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.170456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.171599 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.171602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.172353 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.172354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.172425 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.172429 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.172505 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.172568 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":44,"thread_start_us":109,"threads_started":1}
I20260928 23:10:30.172859 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.169085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:50497 (local address 127.15.212.67:43635)
0928 23:10:30.169135 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.169137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.169147 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:30.169173 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.169174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.169184 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.169251 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.169533 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.170303 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.171689 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.172267 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.172341 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.172349 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.172724 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.172782 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 23:10:30.173230 1004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90a3a35b41447398a43a6c54d76232" candidate_uuid: "d31e3e5a8114461ebdbf4a20675ca308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de261b4509ab4c0099ca8e00969ea588" is_pre_election: true
I20260928 23:10:30.173374 1004 raft_consensus.cc:2454] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31e3e5a8114461ebdbf4a20675ca308 in term 0.
I20260928 23:10:30.173593 1138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90a3a35b41447398a43a6c54d76232" candidate_uuid: "d31e3e5a8114461ebdbf4a20675ca308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cad91a86b641d8968af2c39711c80f" is_pre_election: true
I20260928 23:10:30.173640 803 leader_election.cc:302] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d31e3e5a8114461ebdbf4a20675ca308, de261b4509ab4c0099ca8e00969ea588; no voters:
I20260928 23:10:30.173732 1138 raft_consensus.cc:2454] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d31e3e5a8114461ebdbf4a20675ca308 in term 0.
I20260928 23:10:30.173756 1208 raft_consensus.cc:2781] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:30.173810 1208 raft_consensus.cc:482] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:30.173844 1208 raft_consensus.cc:3037] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:30.174542 1208 raft_consensus.cc:504] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.174641 1208 leader_election.cc:288] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [CANDIDATE]: Term 1 election: Requested vote from peers a1cad91a86b641d8968af2c39711c80f (127.15.212.67:43635), de261b4509ab4c0099ca8e00969ea588 (127.15.212.66:33245)
I20260928 23:10:30.174908 1138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90a3a35b41447398a43a6c54d76232" candidate_uuid: "d31e3e5a8114461ebdbf4a20675ca308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1cad91a86b641d8968af2c39711c80f"
I20260928 23:10:30.174999 1138 raft_consensus.cc:3037] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:30.175684 1138 raft_consensus.cc:2454] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31e3e5a8114461ebdbf4a20675ca308 in term 1.
I20260928 23:10:30.175812 1004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a90a3a35b41447398a43a6c54d76232" candidate_uuid: "d31e3e5a8114461ebdbf4a20675ca308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de261b4509ab4c0099ca8e00969ea588"
I20260928 23:10:30.175891 1004 raft_consensus.cc:3037] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:30.175916 803 leader_election.cc:302] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1cad91a86b641d8968af2c39711c80f, d31e3e5a8114461ebdbf4a20675ca308; no voters:
I20260928 23:10:30.176008 1208 raft_consensus.cc:2781] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:30.176200 1208 raft_consensus.cc:686] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [term 1 LEADER]: Becoming Leader. State: Replica: d31e3e5a8114461ebdbf4a20675ca308, State: Running, Role: LEADER
I20260928 23:10:30.176298 1208 consensus_queue.cc:237] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } }
I20260928 23:10:30.176807 1004 raft_consensus.cc:2454] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d31e3e5a8114461ebdbf4a20675ca308 in term 1.
I20260928 23:10:30.177107 725 catalog_manager.cc:5495] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 reported cstate change: term changed from 0 to 1, leader changed from <none> to d31e3e5a8114461ebdbf4a20675ca308 (127.15.212.65). New cstate: current_term: 1 leader_uuid: "d31e3e5a8114461ebdbf4a20675ca308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d31e3e5a8114461ebdbf4a20675ca308" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 } health_report { overall_health: UNKNOWN } } }
I20260928 23:10:30.187620 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.185247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.15.212.65:45155)
0928 23:10:30.185303 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.185305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.185312 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:10:30.185401 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.185407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.185423 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.185497 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:30.185632 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.186078 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.186865 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.187057 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.187141 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.187306 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:30.187400 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:30.187491 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.187511 (+ 20us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.187531 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260928 23:10:30.189071 1051 tablet.cc:2369] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:30.189267 1004 raft_consensus.cc:1261] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 1 FOLLOWER]: Refusing update from remote peer d31e3e5a8114461ebdbf4a20675ca308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:30.189276 1138 raft_consensus.cc:1261] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 1 FOLLOWER]: Refusing update from remote peer d31e3e5a8114461ebdbf4a20675ca308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:30.189606 1208 consensus_queue.cc:1035] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1cad91a86b641d8968af2c39711c80f" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:30.189743 1211 consensus_queue.cc:1035] T 3a90a3a35b41447398a43a6c54d76232 P d31e3e5a8114461ebdbf4a20675ca308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de261b4509ab4c0099ca8e00969ea588" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:30.191679 1214 mvcc.cc:205] Tried to move back new op lower bound from 7334449275652710400 to 7334449275602825216. Current Snapshot: MvccSnapshot[applied={T|T < 7334449275652710400}]
I20260928 23:10:30.192518 1215 mvcc.cc:205] Tried to move back new op lower bound from 7334449275652710400 to 7334449275602825216. Current Snapshot: MvccSnapshot[applied={T|T < 7334449275652710400}]
I20260928 23:10:30.192991 1216 mvcc.cc:205] Tried to move back new op lower bound from 7334449275652710400 to 7334449275602825216. Current Snapshot: MvccSnapshot[applied={T|T < 7334449275652710400}]
I20260928 23:10:30.193969 724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:30.194068 724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:30.194818 724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e32643711ba4851b4ff120c8dd8ecda: Sending DeleteTablet for 3 replicas of tablet 3a90a3a35b41447398a43a6c54d76232
I20260928 23:10:30.195122 1118 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90a3a35b41447398a43a6c54d76232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:30 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:39028
I20260928 23:10:30.195133 850 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90a3a35b41447398a43a6c54d76232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:30 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:46396
I20260928 23:10:30.195194 984 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a90a3a35b41447398a43a6c54d76232 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:30 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:37284
I20260928 23:10:30.195318 1223 tablet_replica.cc:330] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: stopping tablet replica
I20260928 23:10:30.195447 1223 raft_consensus.cc:2229] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:30.195413 1225 tablet_replica.cc:330] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: stopping tablet replica
I20260928 23:10:30.195472 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 787
I20260928 23:10:30.196204 1223 raft_consensus.cc:2258] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:30.197078 1225 raft_consensus.cc:2229] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:30.197209 1225 raft_consensus.cc:2258] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:30.197609 1223 ts_tablet_manager.cc:1905] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:30.197621 1225 ts_tablet_manager.cc:1905] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:30.199294 1225 ts_tablet_manager.cc:1918] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:30.199368 1223 ts_tablet_manager.cc:1918] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:30.199396 1225 log.cc:1196] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/3a90a3a35b41447398a43a6c54d76232
I20260928 23:10:30.199440 1223 log.cc:1196] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/3a90a3a35b41447398a43a6c54d76232
I20260928 23:10:30.199635 1225 ts_tablet_manager.cc:1939] T 3a90a3a35b41447398a43a6c54d76232 P de261b4509ab4c0099ca8e00969ea588: Deleting consensus metadata
I20260928 23:10:30.199713 1223 ts_tablet_manager.cc:1939] T 3a90a3a35b41447398a43a6c54d76232 P a1cad91a86b641d8968af2c39711c80f: Deleting consensus metadata
I20260928 23:10:30.199966 709 catalog_manager.cc:4841] TS de261b4509ab4c0099ca8e00969ea588 (127.15.212.66:33245): tablet 3a90a3a35b41447398a43a6c54d76232 (table test-table [id=bfc35cc705094e88b1360328539debcb]) successfully deleted
I20260928 23:10:30.200101 709 catalog_manager.cc:4841] TS a1cad91a86b641d8968af2c39711c80f (127.15.212.67:43635): tablet 3a90a3a35b41447398a43a6c54d76232 (table test-table [id=bfc35cc705094e88b1360328539debcb]) successfully deleted
W20260928 23:10:30.202230 710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:45155 (error 108)
I20260928 23:10:30.202471 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 921
I20260928 23:10:30.203275 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.202643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:45155 (local address 127.0.0.1:46418)
0928 23:10:30.203196 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.203234 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:45155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536}
W20260928 23:10:30.204246 710 catalog_manager.cc:4584] TS d31e3e5a8114461ebdbf4a20675ca308 (127.15.212.65:45155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a90a3a35b41447398a43a6c54d76232: Network error: Client connection negotiation failed: client connection to 127.15.212.65:45155: connect: Connection refused (error 111)
I20260928 23:10:30.206954 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1055
I20260928 23:10:30.211337 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 693
2026-09-28T23:10:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T23:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:30Z Disabled control of system clock
I20260928 23:10:30.238206 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40239
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:40239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:30.316413 1230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:30.316636 1230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:30.316656 1230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:30.317878 1230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:30.317919 1230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:30.317932 1230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:30.317945 1230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:30.317956 1230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:30.319221 1230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:40239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:30.319406 1230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:30.319633 1230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:30.324234 1235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.324290 1230 server_base.cc:1034] running on GCE node
W20260928 23:10:30.324241 1238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.324239 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.324703 1230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:30.325021 1230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:30.326186 1230 hybrid_clock.cc:648] HybridClock initialized: now 1790637030326180 us; error 34 us; skew 500 ppm
I20260928 23:10:30.327320 1230 webserver.cc:460] Webserver started at http://127.15.212.126:37121/ using document root <none> and password file <none>
I20260928 23:10:30.327553 1230 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:30.327623 1230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:30.327775 1230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:30.328846 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "3e588b2fb14a41e0b304e37eef20dc45"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "77587216455978035467b6390c850963"
server_key_iv: "ae079523e9a3aeb2ff6ddc6705072c8c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.329200 1230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "3e588b2fb14a41e0b304e37eef20dc45"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "77587216455978035467b6390c850963"
server_key_iv: "ae079523e9a3aeb2ff6ddc6705072c8c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.330426 1230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:10:30.331112 1244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.331282 1230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.331388 1230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "3e588b2fb14a41e0b304e37eef20dc45"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "77587216455978035467b6390c850963"
server_key_iv: "ae079523e9a3aeb2ff6ddc6705072c8c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.331465 1230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:30.364574 1230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:30.364943 1230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:30.365113 1230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:30.370281 1230 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:40239
I20260928 23:10:30.370332 1296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:40239 every 8 connection(s)
I20260928 23:10:30.370692 1230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:30.371788 1297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.372624 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1230
I20260928 23:10:30.372722 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:30.372944 16209 external_mini_cluster.cc:1428] Setting key 5d72583c6f7352297e4d9c1326af2349
I20260928 23:10:30.375120 1297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: Bootstrap starting.
I20260928 23:10:30.375805 1297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.376087 1297 log.cc:824] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.376780 1297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: No bootstrap required, opened a new log
I20260928 23:10:30.379048 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.373460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.15.212.126:40239)
0928 23:10:30.373672 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.373682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.373715 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:30.374280 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.374290 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.374383 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.374532 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.374708 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.376051 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.376895 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.377517 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.377641 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.378293 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.378297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.378298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:30.378502 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.378549 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:30.378564 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.378627 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.378690 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.378780 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260928 23:10:30.379333 1297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } }
I20260928 23:10:30.379516 1297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.379551 1297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e588b2fb14a41e0b304e37eef20dc45, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.379665 1297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } }
I20260928 23:10:30.379799 1297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:30.379849 1297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:30.379894 1297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:30.380362 1297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } }
I20260928 23:10:30.380523 1297 leader_election.cc:302] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e588b2fb14a41e0b304e37eef20dc45; no voters:
I20260928 23:10:30.380672 1297 leader_election.cc:288] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:30.380748 1302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:30.380888 1302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [term 1 LEADER]: Becoming Leader. State: Replica: 3e588b2fb14a41e0b304e37eef20dc45, State: Running, Role: LEADER
I20260928 23:10:30.380926 1297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:30.380975 1302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } }
I20260928 23:10:30.381508 1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e588b2fb14a41e0b304e37eef20dc45. Latest consensus state: current_term: 1 leader_uuid: "3e588b2fb14a41e0b304e37eef20dc45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } } }
I20260928 23:10:30.381521 1303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e588b2fb14a41e0b304e37eef20dc45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e588b2fb14a41e0b304e37eef20dc45" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40239 } } }
I20260928 23:10:30.381615 1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:30.381623 1303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:30.381815 1311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:30.382330 1311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:30.383810 1311 catalog_manager.cc:1347] Generated new cluster ID: 0f14b3c535c74338ab688bbe834560d6
I20260928 23:10:30.383869 1311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:30.404475 1311 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:30.405014 1311 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:30.408377 1311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: Generated new TSK 0
I20260928 23:10:30.408577 1311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:30.412876 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--builtin_ntp_servers=127.15.212.84:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:30.493114 1321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:30.493336 1321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:30.493371 1321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:30.494514 1321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:30.494583 1321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:30.494623 1321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:30.495946 1321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:30.496196 1321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:30.496536 1321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:30.502072 1326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.502060 1329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.502059 1327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.502388 1321 server_base.cc:1034] running on GCE node
I20260928 23:10:30.502569 1321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:30.502748 1321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:30.503891 1321 hybrid_clock.cc:648] HybridClock initialized: now 1790637030503873 us; error 30 us; skew 500 ppm
I20260928 23:10:30.505131 1321 webserver.cc:460] Webserver started at http://127.15.212.65:34533/ using document root <none> and password file <none>
I20260928 23:10:30.505319 1321 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:30.505359 1321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:30.505447 1321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:30.506423 1321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "6c3836361a9048fea744ba1aadba4147"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "cd84c65f1e9c83ef511f93d098ddcb38"
server_key_iv: "80ef859504cbd0a07b35c990f76c587b"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.506724 1321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "6c3836361a9048fea744ba1aadba4147"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "cd84c65f1e9c83ef511f93d098ddcb38"
server_key_iv: "80ef859504cbd0a07b35c990f76c587b"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.507917 1321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:30.508630 1335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.508786 1321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:30.508853 1321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "6c3836361a9048fea744ba1aadba4147"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "cd84c65f1e9c83ef511f93d098ddcb38"
server_key_iv: "80ef859504cbd0a07b35c990f76c587b"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.508903 1321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:30.518977 1321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:30.519191 1321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:30.519284 1321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:30.519434 1321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:30.519683 1321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:30.519712 1321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.519730 1321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:30.519743 1321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.525399 1321 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:45845
I20260928 23:10:30.525461 1448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:45845 every 8 connection(s)
I20260928 23:10:30.525781 1321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:30.527390 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1321
I20260928 23:10:30.527487 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:30.527732 16209 external_mini_cluster.cc:1428] Setting key e7aeec7534b6a9c57b35b9fab2f7e112
I20260928 23:10:30.528738 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--builtin_ntp_servers=127.15.212.84:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 23:10:30.531320 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.526414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:58789 (local address 127.15.212.126:40239)
0928 23:10:30.526478 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.526480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.526695 (+ 215us) server_negotiation.cc:407] Connection header received
0928 23:10:30.527158 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.527159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.527170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.527239 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.528205 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.528867 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.529864 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.530064 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.530151 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.531046 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.531047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.531048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:30.531052 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.531067 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:30.531071 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.531118 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.531209 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.531243 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 23:10:30.531524 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.526418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40239 (local address 127.15.212.65:58789)
0928 23:10:30.526585 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.526616 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.527076 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.527348 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.527353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.527365 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:30.528107 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.528114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.528982 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.528982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.529753 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.529756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.530163 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.530164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.530289 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.530880 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:30.530881 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:30.530909 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:30.530932 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:30.531118 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:30.531123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:30.531126 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.531211 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.531307 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":92,"threads_started":1}
I20260928 23:10:30.531987 1449 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40239
I20260928 23:10:30.532104 1449 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:30.532271 1449 heartbeater.cc:511] Master 127.15.212.126:40239 requested a full tablet report, sending...
I20260928 23:10:30.532796 1261 ts_manager.cc:194] Registered new tserver with Master: 6c3836361a9048fea744ba1aadba4147 (127.15.212.65:45845)
I20260928 23:10:30.533397 1261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.65:58789
W20260928 23:10:30.608021 1452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:30.608256 1452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:30.608290 1452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:30.609493 1452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:30.609565 1452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:30.609604 1452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:30.610982 1452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:30.611258 1452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:30.611552 1452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:10:30.616595 1452 server_base.cc:1034] running on GCE node
W20260928 23:10:30.616567 1458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.616544 1457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.616567 1460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.617007 1452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:30.617236 1452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:30.618381 1452 hybrid_clock.cc:648] HybridClock initialized: now 1790637030618370 us; error 25 us; skew 500 ppm
I20260928 23:10:30.619660 1452 webserver.cc:460] Webserver started at http://127.15.212.66:40651/ using document root <none> and password file <none>
I20260928 23:10:30.619896 1452 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:30.619944 1452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:30.620085 1452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:30.621171 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "977a63e6a62f483cb228d68273688c6e"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "343775df614abf7d493fc44bc5f9eb89"
server_key_iv: "f389ba30cd7ef365ab5530ef9a8ef118"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.621529 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "977a63e6a62f483cb228d68273688c6e"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "343775df614abf7d493fc44bc5f9eb89"
server_key_iv: "f389ba30cd7ef365ab5530ef9a8ef118"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.622799 1452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:30.623513 1466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.623700 1452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:30.623790 1452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "977a63e6a62f483cb228d68273688c6e"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "343775df614abf7d493fc44bc5f9eb89"
server_key_iv: "f389ba30cd7ef365ab5530ef9a8ef118"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.623862 1452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:30.640422 1452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:30.640746 1452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:30.640892 1452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:30.641114 1452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:30.641417 1452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:30.641449 1452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.641469 1452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:30.641512 1452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.647022 1452 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:44089
I20260928 23:10:30.647089 1579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:44089 every 8 connection(s)
I20260928 23:10:30.647395 1452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:10:30.652371 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.648106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:53837 (local address 127.15.212.126:40239)
0928 23:10:30.648161 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.648163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.648392 (+ 229us) server_negotiation.cc:407] Connection header received
0928 23:10:30.648966 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.648968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.648980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.649037 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.649665 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.650150 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.651083 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.651276 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.651357 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.652081 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.652083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.652083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:30.652088 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.652103 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:30.652107 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.652169 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.652254 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.652280 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:10:30.652585 1582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.648102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40239 (local address 127.15.212.66:53837)
0928 23:10:30.648281 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.648312 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.648842 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.649109 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.649113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.649120 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:30.649575 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.649579 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.650242 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.650243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.650977 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.650978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.651367 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.651368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.651490 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.651939 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:30.651939 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:30.651974 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:30.651995 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:30.652173 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:30.652178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:30.652181 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.652255 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.652364 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260928 23:10:30.653016 1580 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40239
I20260928 23:10:30.653098 1580 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:30.653223 1580 heartbeater.cc:511] Master 127.15.212.126:40239 requested a full tablet report, sending...
I20260928 23:10:30.653493 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1452
I20260928 23:10:30.653594 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:30.653614 1261 ts_manager.cc:194] Registered new tserver with Master: 977a63e6a62f483cb228d68273688c6e (127.15.212.66:44089)
I20260928 23:10:30.653829 16209 external_mini_cluster.cc:1428] Setting key 1e1d5ff54b6095576315ee61efd3c1a3
I20260928 23:10:30.654109 1261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.66:53837
I20260928 23:10:30.654917 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--builtin_ntp_servers=127.15.212.84:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:30.732898 1583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:30.733135 1583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:30.733170 1583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:30.734383 1583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:30.734443 1583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:30.734484 1583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:30.735818 1583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:30.736070 1583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:30.736356 1583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:30.741180 1588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.741189 1589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:30.741183 1591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:30.741465 1583 server_base.cc:1034] running on GCE node
I20260928 23:10:30.741645 1583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:30.741818 1583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:30.742981 1583 hybrid_clock.cc:648] HybridClock initialized: now 1790637030742957 us; error 33 us; skew 500 ppm
I20260928 23:10:30.744130 1583 webserver.cc:460] Webserver started at http://127.15.212.67:40271/ using document root <none> and password file <none>
I20260928 23:10:30.744371 1583 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:30.744438 1583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:30.744622 1583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:30.745610 1583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "24ff53a0056c45e1b0b288586eda0c81"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "bbd0607208889bba9d0380c43c2f56ec"
server_key_iv: "0c7aae39383fe11f81e051cdcc1c225c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.745951 1583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "24ff53a0056c45e1b0b288586eda0c81"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "bbd0607208889bba9d0380c43c2f56ec"
server_key_iv: "0c7aae39383fe11f81e051cdcc1c225c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.747175 1583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:10:30.747848 1597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.748013 1583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:30.748111 1583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "24ff53a0056c45e1b0b288586eda0c81"
format_stamp: "Formatted at 2026-09-28 23:10:30 on dist-test-slave-x2t8"
server_key: "bbd0607208889bba9d0380c43c2f56ec"
server_key_iv: "0c7aae39383fe11f81e051cdcc1c225c"
server_key_version: "encryptionkey@0"
I20260928 23:10:30.748189 1583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:30.757978 1583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:30.758229 1583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:30.758368 1583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:30.758584 1583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:30.758880 1583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:30.758937 1583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.758975 1583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:30.759003 1583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:30.764391 1583 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:40035
I20260928 23:10:30.764432 1710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:40035 every 8 connection(s)
I20260928 23:10:30.764752 1583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:10:30.769127 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1583
I20260928 23:10:30.769243 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:30.769450 16209 external_mini_cluster.cc:1428] Setting key 91fa4a5822a2b190b729aaee16057cc6
I20260928 23:10:30.769445 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.765429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:42511 (local address 127.15.212.126:40239)
0928 23:10:30.765508 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.765510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.765721 (+ 211us) server_negotiation.cc:407] Connection header received
0928 23:10:30.766137 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.766138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.766148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.766198 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.766790 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.767258 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.768153 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.768321 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.768368 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.769109 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.769110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.769111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:30.769116 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.769133 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:30.769138 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.769201 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.769344 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.769367 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:10:30.769647 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.765406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40239 (local address 127.15.212.67:42511)
0928 23:10:30.765606 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.765637 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.766057 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.766245 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.766247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.766254 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:30.766720 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.766724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.767331 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.767332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.768078 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.768080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.768388 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.768388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.768509 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.768976 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:30.768976 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:30.769001 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:30.769022 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:30.769226 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:30.769231 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:30.769234 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.769310 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.769406 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260928 23:10:30.770080 1711 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40239
I20260928 23:10:30.770170 1711 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:30.770344 1711 heartbeater.cc:511] Master 127.15.212.126:40239 requested a full tablet report, sending...
I20260928 23:10:30.770685 1260 ts_manager.cc:194] Registered new tserver with Master: 24ff53a0056c45e1b0b288586eda0c81 (127.15.212.67:40035)
I20260928 23:10:30.771010 1260 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.67:42511
I20260928 23:10:30.781378 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:30.785597 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.782779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.15.212.126:40239)
0928 23:10:30.782847 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.782849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.782923 (+ 74us) server_negotiation.cc:407] Connection header received
0928 23:10:30.782988 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.782989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.782999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.783052 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:30.783223 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.783700 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.784465 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.784672 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.784756 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.785350 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:30.785351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:30.785351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:30.785355 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:30.785368 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:30.785372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:30.785439 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.785498 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.785518 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 23:10:30.786916 1260 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:30.788363 1260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:30.794754 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:45845 (local address 127.0.0.1:56662)
0928 23:10:30.790561 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.790601 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.790659 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.791264 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.791267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.791269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.791339 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.791343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.792281 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.792282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.793526 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.793528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.794427 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.794428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.794533 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.794554 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.794620 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.794672 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":82,"threads_started":1}
I20260928 23:10:30.794852 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.15.212.66:44089)
0928 23:10:30.791099 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.791104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.791114 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:30.791156 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.791158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.791174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.791250 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.791445 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.792562 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.793773 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.794455 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.794546 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.794623 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.794734 (+ 111us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.794771 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
I20260928 23:10:30.795082 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56662 (local address 127.15.212.65:45845)
0928 23:10:30.791088 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.791093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.791103 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:30.791138 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.791139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.791153 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.791417 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.791477 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.792191 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.793608 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.794315 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.794410 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.794526 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.794945 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.794997 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":74,"threads_started":1}
I20260928 23:10:30.795197 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:44089 (local address 127.0.0.1:50242)
0928 23:10:30.790859 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.790886 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.790942 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.791317 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.791318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.791320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.791364 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.791365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.792659 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.792660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.793701 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.793702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.794609 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.794610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.794657 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.794660 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.794708 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.794739 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260928 23:10:30.795750 1383 tablet_service.cc:1467] Processing CreateTablet for tablet 252bed0f51574d6890ea7d880a0833c1 (DEFAULT_TABLE table=test-table [id=f581930695b243ba9cf7585e08b31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.795818 1514 tablet_service.cc:1467] Processing CreateTablet for tablet 252bed0f51574d6890ea7d880a0833c1 (DEFAULT_TABLE table=test-table [id=f581930695b243ba9cf7585e08b31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.796097 1383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252bed0f51574d6890ea7d880a0833c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.796132 1514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252bed0f51574d6890ea7d880a0833c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.796940 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40035 (local address 127.0.0.1:50494)
0928 23:10:30.790788 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:30.790798 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:30.790850 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:30.791296 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:30.791296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:30.791298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:30.791339 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.791341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.793767 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.793767 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.794456 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:30.794457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.796697 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.796699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:30.796764 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.796768 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:30.796835 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 23:10:30.796876 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
I20260928 23:10:30.796962 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:30.790627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.15.212.67:40035)
0928 23:10:30.790767 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:30.790772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:30.791084 (+ 312us) server_negotiation.cc:407] Connection header received
0928 23:10:30.791132 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:30.791134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:30.791152 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:30.791231 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:30.791408 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.792562 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.794585 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:30.796607 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:30.796683 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:30.796766 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:30.796847 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 23:10:30.796883 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260928 23:10:30.797847 1645 tablet_service.cc:1467] Processing CreateTablet for tablet 252bed0f51574d6890ea7d880a0833c1 (DEFAULT_TABLE table=test-table [id=f581930695b243ba9cf7585e08b31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:30.798152 1645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252bed0f51574d6890ea7d880a0833c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:30.799053 1725 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Bootstrap starting.
I20260928 23:10:30.799396 1726 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Bootstrap starting.
I20260928 23:10:30.799854 1725 tablet_bootstrap.cc:654] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.800028 1726 tablet_bootstrap.cc:654] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.800186 1725 log.cc:824] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.800323 1726 log.cc:824] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.800977 1725 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: No bootstrap required, opened a new log
I20260928 23:10:30.801066 1725 ts_tablet_manager.cc:1397] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:30.801069 1726 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: No bootstrap required, opened a new log
I20260928 23:10:30.801133 1726 ts_tablet_manager.cc:1397] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:30.801676 1729 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Bootstrap starting.
I20260928 23:10:30.802320 1729 tablet_bootstrap.cc:654] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:30.802642 1729 log.cc:824] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:30.803486 1725 raft_consensus.cc:348] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.803531 1726 raft_consensus.cc:348] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.803671 1729 tablet_bootstrap.cc:492] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: No bootstrap required, opened a new log
I20260928 23:10:30.803661 1725 raft_consensus.cc:374] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.803682 1726 raft_consensus.cc:374] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.803704 1725 raft_consensus.cc:729] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3836361a9048fea744ba1aadba4147, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.803722 1729 ts_tablet_manager.cc:1397] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:30.803727 1726 raft_consensus.cc:729] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977a63e6a62f483cb228d68273688c6e, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.803845 1726 consensus_queue.cc:260] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.803840 1725 consensus_queue.cc:260] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.804131 1726 ts_tablet_manager.cc:1428] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:10:30.804134 1725 ts_tablet_manager.cc:1428] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:10:30.804251 1449 heartbeater.cc:503] Master 127.15.212.126:40239 was elected leader, sending a full tablet report...
I20260928 23:10:30.804251 1580 heartbeater.cc:503] Master 127.15.212.126:40239 was elected leader, sending a full tablet report...
I20260928 23:10:30.806311 1729 raft_consensus.cc:348] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.806425 1729 raft_consensus.cc:374] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:30.806447 1729 raft_consensus.cc:729] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ff53a0056c45e1b0b288586eda0c81, State: Initialized, Role: FOLLOWER
I20260928 23:10:30.806552 1729 consensus_queue.cc:260] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:30.806761 1729 ts_tablet_manager.cc:1428] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 23:10:30.806830 1711 heartbeater.cc:503] Master 127.15.212.126:40239 was elected leader, sending a full tablet report...
W20260928 23:10:30.897984 1581 tablet.cc:2369] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:31.015262 1712 tablet.cc:2369] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:31.026501 1450 tablet.cc:2369] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:31.030802 1732 raft_consensus.cc:482] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:31.030954 1732 raft_consensus.cc:504] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:31.031349 1732 leader_election.cc:288] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c3836361a9048fea744ba1aadba4147 (127.15.212.65:45845), 24ff53a0056c45e1b0b288586eda0c81 (127.15.212.67:40035)
I20260928 23:10:31.034441 1582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.031505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:45845 (local address 127.15.212.66:51125)
0928 23:10:31.031549 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:31.031561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:31.031609 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:31.031774 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:31.031775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:31.031777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:31.031857 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:31.031859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.032609 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.032610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:31.033437 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:31.033440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.034123 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.034124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:31.034205 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:31.034210 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:10:31.034273 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 23:10:31.034354 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":26}
I20260928 23:10:31.034471 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.031637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:53379 (local address 127.15.212.67:40035)
0928 23:10:31.031679 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:31.031682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:31.031792 (+ 110us) server_negotiation.cc:407] Connection header received
0928 23:10:31.031845 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:31.031846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:31.031853 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:31.031891 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:31.032049 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.032612 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.033571 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.034118 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.034177 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:31.034190 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:31.034358 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 23:10:31.034403 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:10:31.034471 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.031578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:51125 (local address 127.15.212.65:45845)
0928 23:10:31.031627 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:31.031629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:31.031639 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:31.031670 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:31.031671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:31.031684 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:31.031746 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:31.031984 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.032536 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.033536 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.034051 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.034127 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:31.034136 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:31.034358 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 23:10:31.034403 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 23:10:31.034448 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.031540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40035 (local address 127.15.212.66:53379)
0928 23:10:31.031717 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:31.031749 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:31.031793 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:31.031913 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:31.031913 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:31.031915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:31.031974 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:31.031975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.032694 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.032694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:31.033500 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:31.033503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.034184 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.034185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:31.034263 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:31.034267 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:31.034315 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 23:10:31.034360 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":22,"thread_start_us":111,"threads_started":1}
I20260928 23:10:31.034785 1403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252bed0f51574d6890ea7d880a0833c1" candidate_uuid: "977a63e6a62f483cb228d68273688c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c3836361a9048fea744ba1aadba4147" is_pre_election: true
I20260928 23:10:31.034806 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252bed0f51574d6890ea7d880a0833c1" candidate_uuid: "977a63e6a62f483cb228d68273688c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ff53a0056c45e1b0b288586eda0c81" is_pre_election: true
I20260928 23:10:31.034935 1403 raft_consensus.cc:2454] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977a63e6a62f483cb228d68273688c6e in term 0.
I20260928 23:10:31.034969 1665 raft_consensus.cc:2454] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977a63e6a62f483cb228d68273688c6e in term 0.
I20260928 23:10:31.035207 1467 leader_election.cc:302] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ff53a0056c45e1b0b288586eda0c81, 977a63e6a62f483cb228d68273688c6e; no voters:
I20260928 23:10:31.035338 1732 raft_consensus.cc:2781] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:31.035391 1732 raft_consensus.cc:482] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:31.035414 1732 raft_consensus.cc:3037] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:31.036101 1732 raft_consensus.cc:504] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:31.036238 1732 leader_election.cc:288] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [CANDIDATE]: Term 1 election: Requested vote from peers 6c3836361a9048fea744ba1aadba4147 (127.15.212.65:45845), 24ff53a0056c45e1b0b288586eda0c81 (127.15.212.67:40035)
I20260928 23:10:31.036489 1665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252bed0f51574d6890ea7d880a0833c1" candidate_uuid: "977a63e6a62f483cb228d68273688c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ff53a0056c45e1b0b288586eda0c81"
I20260928 23:10:31.036548 1665 raft_consensus.cc:3037] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:31.037246 1665 raft_consensus.cc:2454] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977a63e6a62f483cb228d68273688c6e in term 1.
I20260928 23:10:31.036398 1403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252bed0f51574d6890ea7d880a0833c1" candidate_uuid: "977a63e6a62f483cb228d68273688c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c3836361a9048fea744ba1aadba4147"
I20260928 23:10:31.037437 1403 raft_consensus.cc:3037] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:31.037439 1467 leader_election.cc:302] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ff53a0056c45e1b0b288586eda0c81, 977a63e6a62f483cb228d68273688c6e; no voters:
I20260928 23:10:31.037565 1732 raft_consensus.cc:2781] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:31.037711 1732 raft_consensus.cc:686] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 1 LEADER]: Becoming Leader. State: Replica: 977a63e6a62f483cb228d68273688c6e, State: Running, Role: LEADER
I20260928 23:10:31.037823 1732 consensus_queue.cc:237] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } }
I20260928 23:10:31.038344 1403 raft_consensus.cc:2454] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977a63e6a62f483cb228d68273688c6e in term 1.
I20260928 23:10:31.038609 1260 catalog_manager.cc:5495] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 977a63e6a62f483cb228d68273688c6e (127.15.212.66). New cstate: current_term: 1 leader_uuid: "977a63e6a62f483cb228d68273688c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "977a63e6a62f483cb228d68273688c6e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260928 23:10:31.077349 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.074824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.15.212.66:44089)
0928 23:10:31.074865 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:31.074866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:31.074876 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:31.074915 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:31.074915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:31.074932 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:31.074993 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:31.075087 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.075557 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.076522 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:31.076709 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:31.076762 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:31.076901 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:31.077033 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:31.077151 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:31.077170 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 23:10:31.077195 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:10:31.079720 1403 raft_consensus.cc:1261] T 252bed0f51574d6890ea7d880a0833c1 P 6c3836361a9048fea744ba1aadba4147 [term 1 FOLLOWER]: Refusing update from remote peer 977a63e6a62f483cb228d68273688c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:31.080328 1732 consensus_queue.cc:1035] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c3836361a9048fea744ba1aadba4147" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:31.083009 1738 mvcc.cc:205] Tried to move back new op lower bound from 7334449279299387392 to 7334449279131414528. Current Snapshot: MvccSnapshot[applied={T|T < 7334449279299387392}]
I20260928 23:10:31.085018 1665 raft_consensus.cc:1261] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 1 FOLLOWER]: Refusing update from remote peer 977a63e6a62f483cb228d68273688c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:31.085451 1735 consensus_queue.cc:1035] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24ff53a0056c45e1b0b288586eda0c81" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:31.087287 1261 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:31.087368 1261 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:31.088079 1261 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e588b2fb14a41e0b304e37eef20dc45: Sending DeleteTablet for 3 replicas of tablet 252bed0f51574d6890ea7d880a0833c1
I20260928 23:10:31.088091 1744 mvcc.cc:205] Tried to move back new op lower bound from 7334449279299387392 to 7334449279131414528. Current Snapshot: MvccSnapshot[applied={T|T < 7334449279299387392}]
I20260928 23:10:31.088784 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1321
I20260928 23:10:31.088588 1645 tablet_service.cc:1514] Processing DeleteTablet for tablet 252bed0f51574d6890ea7d880a0833c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:31 UTC) from {username='slave'} at 127.0.0.1:50494
I20260928 23:10:31.090133 1514 tablet_service.cc:1514] Processing DeleteTablet for tablet 252bed0f51574d6890ea7d880a0833c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:31 UTC) from {username='slave'} at 127.0.0.1:50242
W20260928 23:10:31.094820 1248 connection.cc:535] client connection to 127.15.212.65:45845 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 23:10:31.094935 1248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 23:10:31.095098 1747 tablet_replica.cc:330] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: stopping tablet replica
I20260928 23:10:31.095206 1747 raft_consensus.cc:2229] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:10:31.095489 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.095336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:45845 (local address 127.0.0.1:56678)
0928 23:10:31.095406 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:31.095455 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:45845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260928 23:10:31.095598 1747 raft_consensus.cc:2258] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 23:10:31.095623 1248 catalog_manager.cc:4584] TS 6c3836361a9048fea744ba1aadba4147 (127.15.212.65:45845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 252bed0f51574d6890ea7d880a0833c1: Network error: Client connection negotiation failed: client connection to 127.15.212.65:45845: connect: Connection refused (error 111)
I20260928 23:10:31.095796 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1452
I20260928 23:10:31.096053 1747 ts_tablet_manager.cc:1905] T 252bed0f51574d6890ea7d880a0833c1 P 977a63e6a62f483cb228d68273688c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:31.097010 1748 tablet_replica.cc:330] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: stopping tablet replica
I20260928 23:10:31.097103 1748 raft_consensus.cc:2229] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:31.097205 1748 raft_consensus.cc:2258] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:31.097631 1748 ts_tablet_manager.cc:1905] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:31.099529 1748 ts_tablet_manager.cc:1918] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:31.099596 1748 log.cc:1196] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/252bed0f51574d6890ea7d880a0833c1
I20260928 23:10:31.099807 1748 ts_tablet_manager.cc:1939] T 252bed0f51574d6890ea7d880a0833c1 P 24ff53a0056c45e1b0b288586eda0c81: Deleting consensus metadata
I20260928 23:10:31.100214 1245 catalog_manager.cc:4841] TS 24ff53a0056c45e1b0b288586eda0c81 (127.15.212.67:40035): tablet 252bed0f51574d6890ea7d880a0833c1 (table test-table [id=f581930695b243ba9cf7585e08b31724]) successfully deleted
I20260928 23:10:31.101764 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1583
I20260928 23:10:31.102300 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:31.101846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:44089 (local address 127.0.0.1:50266)
0928 23:10:31.102238 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:31.102263 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.66:44089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374}
I20260928 23:10:31.106979 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1230
2026-09-28T23:10:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (897 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:31 dist-test-slave-x2t8 krb5kdc[1752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:31 dist-test-slave-x2t8 krb5kdc[1752](info): set up 2 sockets
Sep 28 23:10:31 dist-test-slave-x2t8 krb5kdc[1752](info): commencing operation
krb5kdc: starting...
W20260928 23:10:32.148697 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:32.264901 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42949
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:42949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:32.345058 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:32.345301 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:32.345337 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:32.346504 1765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:32.346575 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:32.346601 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:32.346632 1765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:32.346657 1765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:32.348166 1765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:42949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:42949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:32.348471 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:32.348712 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:32.353425 1770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.353457 1773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:32.353488 1765 server_base.cc:1034] running on GCE node
W20260928 23:10:32.353435 1771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:32.353801 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:32.354113 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:32.355275 1765 hybrid_clock.cc:648] HybridClock initialized: now 1790637032355261 us; error 30 us; skew 500 ppm
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:32.357739 1765 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:32.358199 1765 webserver.cc:460] Webserver started at http://127.15.212.126:39103/ using document root <none> and password file <none>
I20260928 23:10:32.358428 1765 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:32.358505 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:32.358659 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:32.359689 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "bbeb801261b742ad929fed34d752915a"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "894fc7807a2c47fb368974017ea08684"
server_key_iv: "936a012c363e4180a1dcca807736a126"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.360049 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "bbeb801261b742ad929fed34d752915a"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "894fc7807a2c47fb368974017ea08684"
server_key_iv: "936a012c363e4180a1dcca807736a126"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.361464 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:32.362185 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.362354 1765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:32.362475 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "bbeb801261b742ad929fed34d752915a"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "894fc7807a2c47fb368974017ea08684"
server_key_iv: "936a012c363e4180a1dcca807736a126"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.362552 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:32.384766 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:32.386580 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:32.386747 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:32.390936 1765 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:42949
I20260928 23:10:32.390975 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:42949 every 8 connection(s)
I20260928 23:10:32.391342 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:32.392194 1833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:32.394521 1833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: Bootstrap starting.
I20260928 23:10:32.395115 1833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:32.395402 1833 log.cc:824] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:32.396096 1833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: No bootstrap required, opened a new log
I20260928 23:10:32.397746 1833 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } }
I20260928 23:10:32.397889 1833 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:32.397912 1833 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbeb801261b742ad929fed34d752915a, State: Initialized, Role: FOLLOWER
I20260928 23:10:32.398110 1833 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } }
I20260928 23:10:32.398191 1833 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:32.398233 1833 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:32.398283 1833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:32.398809 1833 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } }
I20260928 23:10:32.398921 1833 leader_election.cc:302] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbeb801261b742ad929fed34d752915a; no voters:
I20260928 23:10:32.399123 1833 leader_election.cc:288] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:32.399183 1836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:32.399379 1836 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [term 1 LEADER]: Becoming Leader. State: Replica: bbeb801261b742ad929fed34d752915a, State: Running, Role: LEADER
I20260928 23:10:32.399438 1833 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:32.399464 1836 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } }
I20260928 23:10:32.399577 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1765
I20260928 23:10:32.399696 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:32.399803 1836 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbeb801261b742ad929fed34d752915a. Latest consensus state: current_term: 1 leader_uuid: "bbeb801261b742ad929fed34d752915a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } } }
I20260928 23:10:32.399878 1836 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:32.399886 16209 external_mini_cluster.cc:1428] Setting key a365edaa50066dd11ca35e2b548aacae
I20260928 23:10:32.399962 1837 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbeb801261b742ad929fed34d752915a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbeb801261b742ad929fed34d752915a" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 42949 } } }
I20260928 23:10:32.400018 1837 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:32.400838 1841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:32.401433 1841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:32.404740 1841 catalog_manager.cc:1347] Generated new cluster ID: aca7b6cb4dd344ac9f1d09640d4dd15e
I20260928 23:10:32.404791 1841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:32.411262 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.400559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.15.212.126:42949)
0928 23:10:32.400732 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.400742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.400780 (+ 38us) server_negotiation.cc:407] Connection header received
0928 23:10:32.401012 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.401021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.401091 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.401256 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:32.401947 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.403102 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.404181 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.404834 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.404941 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.407327 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:32.407331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:32.407331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:32.407343 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:32.409662 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.410055 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.410058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.410058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.410166 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.410404 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.410405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.410405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.410646 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:32.410814 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.410913 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.411057 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260928 23:10:32.416735 1841 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:32.417245 1841 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:32.421912 1841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: Generated new TSK 0
I20260928 23:10:32.422101 1841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:10:32.450953 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:32.550239 1859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:32.550467 1859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:32.550506 1859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:32.551679 1859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:32.551745 1859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:32.551791 1859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:32.553330 1859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:32.553598 1859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:32.553828 1859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:32.558842 1867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.558837 1864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.558837 1865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:32.559424 1859 server_base.cc:1034] running on GCE node
I20260928 23:10:32.559639 1859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:32.559887 1859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:32.561056 1859 hybrid_clock.cc:648] HybridClock initialized: now 1790637032561041 us; error 29 us; skew 500 ppm
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:32.563513 1859 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:10:32.563988 1859 webserver.cc:460] Webserver started at http://127.15.212.65:41869/ using document root <none> and password file <none>
I20260928 23:10:32.564226 1859 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:32.564270 1859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:32.564397 1859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:32.565582 1859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "d03efb01471944ed8f017f32acf59d57"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "8c975840c5974a93b40e73cacfe41afc"
server_key_iv: "c599bdad82baab5f47f73731e6db8673"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.565932 1859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "d03efb01471944ed8f017f32acf59d57"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "8c975840c5974a93b40e73cacfe41afc"
server_key_iv: "c599bdad82baab5f47f73731e6db8673"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.567315 1859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:32.568043 1874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.568226 1859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:32.568318 1859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "d03efb01471944ed8f017f32acf59d57"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "8c975840c5974a93b40e73cacfe41afc"
server_key_iv: "c599bdad82baab5f47f73731e6db8673"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.568370 1859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:32.579551 1859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:32.581452 1859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:32.581573 1859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:32.581751 1859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:32.582065 1859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:32.582099 1859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.582144 1859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:32.582170 1859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.587813 1859 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:36739
I20260928 23:10:32.587896 1987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:36739 every 8 connection(s)
I20260928 23:10:32.588213 1859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:32.596827 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1859
I20260928 23:10:32.596995 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:32.597242 16209 external_mini_cluster.cc:1428] Setting key a6bd726aefbd60b99e2459e0e5ce30d6
I20260928 23:10:32.597780 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.588993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:41179 (local address 127.15.212.126:42949)
0928 23:10:32.589082 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.589084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.589291 (+ 207us) server_negotiation.cc:407] Connection header received
0928 23:10:32.589818 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.589819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.589831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.589922 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:32.590895 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.591412 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.592423 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.592668 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.592776 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.594932 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:32.594934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:32.594934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:32.594942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:32.596182 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.596675 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.596676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.596677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.596735 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.597023 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.597024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.597025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.597186 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:32.597275 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.597584 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.597689 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 23:10:32.597957 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.588930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42949 (local address 127.15.212.65:41179)
0928 23:10:32.589149 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.589191 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.589707 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.590002 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.590006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.590589 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:32.590786 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.590790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.591511 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.591512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.592323 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.592325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.592785 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.592786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.592928 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.593484 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:32.593493 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:32.594832 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:32.596277 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.596284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.596286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.596586 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.596811 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.596812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.596812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.596944 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.597269 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:32.597274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:32.597446 (+ 172us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.597566 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.597729 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
I20260928 23:10:32.599593 1988 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42949
I20260928 23:10:32.599727 1988 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:32.599936 1988 heartbeater.cc:511] Master 127.15.212.126:42949 requested a full tablet report, sending...
I20260928 23:10:32.600585 1797 ts_manager.cc:194] Registered new tserver with Master: d03efb01471944ed8f017f32acf59d57 (127.15.212.65:36739)
I20260928 23:10:32.601514 1797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:41179
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:10:32.622813 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:32.703348 1993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:32.703572 1993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:32.703608 1993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:32.704759 1993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:32.704825 1993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:32.704867 1993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:32.706293 1993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:32.706544 1993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:32.706761 1993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:32.711757 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.711792 1999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.711757 2001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:32.711933 1993 server_base.cc:1034] running on GCE node
I20260928 23:10:32.712206 1993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:32.712410 1993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:32.713588 1993 hybrid_clock.cc:648] HybridClock initialized: now 1790637032713558 us; error 44 us; skew 500 ppm
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:32.715920 1993 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:10:32.716289 1993 webserver.cc:460] Webserver started at http://127.15.212.66:38771/ using document root <none> and password file <none>
I20260928 23:10:32.716549 1993 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:32.716596 1993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:32.716682 1993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:32.717670 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "2b158a0e34d54596b07c52beb56bafe9"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "762a50212195a974a30a60d8b75f12df"
server_key_iv: "a5e16758f1211a61d8a0d47ddd91faef"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.718034 1993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "2b158a0e34d54596b07c52beb56bafe9"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "762a50212195a974a30a60d8b75f12df"
server_key_iv: "a5e16758f1211a61d8a0d47ddd91faef"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.719234 1993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:10:32.719864 2008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.720034 1993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.720134 1993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "2b158a0e34d54596b07c52beb56bafe9"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "762a50212195a974a30a60d8b75f12df"
server_key_iv: "a5e16758f1211a61d8a0d47ddd91faef"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.720211 1993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:32.731140 1993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:32.732895 1993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:32.733052 1993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:32.733268 1993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:32.733564 1993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:32.733616 1993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.733649 1993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:32.733675 1993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.739435 1993 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:43771
I20260928 23:10:32.739486 2121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:43771 every 8 connection(s)
I20260928 23:10:32.739789 1993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:32.747745 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 1993
I20260928 23:10:32.747874 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:32.748152 16209 external_mini_cluster.cc:1428] Setting key 5c007a0b0bbf835e89204af29d7538f5
I20260928 23:10:32.749163 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.740537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:49001 (local address 127.15.212.126:42949)
0928 23:10:32.740623 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.740626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.740822 (+ 196us) server_negotiation.cc:407] Connection header received
0928 23:10:32.741339 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.741340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.741353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.741407 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:32.742345 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.742880 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.743902 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.744109 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.744184 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.746292 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:32.746293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:32.746293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:32.746300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:32.747496 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.748046 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.748047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.748047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.748102 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.748395 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.748396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.748396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.748548 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:32.748624 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.748937 (+ 313us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.749031 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 23:10:32.749243 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.740510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42949 (local address 127.15.212.66:49001)
0928 23:10:32.740701 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.740736 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.741230 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.741497 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.741506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.742055 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:32.742239 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.742243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.742989 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.742990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.743792 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.743794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.744201 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.744203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.744322 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.744899 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:32.744907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:32.746207 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:32.747578 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.747585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.747587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.747925 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.748173 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.748174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.748175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.748314 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.748628 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:32.748635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:32.748771 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.748897 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.749055 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260928 23:10:32.749673 2122 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42949
I20260928 23:10:32.749756 2122 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:32.749953 2122 heartbeater.cc:511] Master 127.15.212.126:42949 requested a full tablet report, sending...
I20260928 23:10:32.750388 1797 ts_manager.cc:194] Registered new tserver with Master: 2b158a0e34d54596b07c52beb56bafe9 (127.15.212.66:43771)
I20260928 23:10:32.750937 1797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:49001
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:10:32.773268 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:32.853178 2127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:32.853363 2127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:32.853386 2127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:32.854547 2127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:32.854588 2127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:32.854660 2127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:32.856052 2127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:42949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:32.856289 2127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:32.856534 2127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:32.861299 2135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.861310 2132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:32.861315 2133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:32.861538 2127 server_base.cc:1034] running on GCE node
I20260928 23:10:32.861737 2127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:32.861972 2127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:32.863148 2127 hybrid_clock.cc:648] HybridClock initialized: now 1790637032863132 us; error 28 us; skew 500 ppm
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:32.865751 2127 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:10:32.866112 2127 webserver.cc:460] Webserver started at http://127.15.212.67:46139/ using document root <none> and password file <none>
I20260928 23:10:32.866283 2127 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:32.866321 2127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:32.866410 2127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:32.867386 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "cb596b38e8294fecb282d35c942dec60"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "03e642143eabc6ab0077b5c3d603a49e"
server_key_iv: "606ab65cdcbd3aabcbb2567406898e35"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.867676 2127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "cb596b38e8294fecb282d35c942dec60"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "03e642143eabc6ab0077b5c3d603a49e"
server_key_iv: "606ab65cdcbd3aabcbb2567406898e35"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.868932 2127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:32.869606 2142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.869776 2127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:32.869887 2127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "cb596b38e8294fecb282d35c942dec60"
format_stamp: "Formatted at 2026-09-28 23:10:32 on dist-test-slave-x2t8"
server_key: "03e642143eabc6ab0077b5c3d603a49e"
server_key_iv: "606ab65cdcbd3aabcbb2567406898e35"
server_key_version: "encryptionkey@0"
I20260928 23:10:32.869959 2127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:32.930053 2127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:32.932452 2127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:32.932639 2127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:32.932883 2127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:32.933205 2127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:32.933262 2127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.933305 2127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:32.933331 2127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:32.938935 2127 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:32993
I20260928 23:10:32.938980 2255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:32993 every 8 connection(s)
I20260928 23:10:32.939327 2127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
Sep 28 23:10:32 dist-test-slave-x2t8 krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:32.948019 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.940065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:34489 (local address 127.15.212.126:42949)
0928 23:10:32.940116 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.940118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.940304 (+ 186us) server_negotiation.cc:407] Connection header received
0928 23:10:32.940837 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.940837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.940846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.940911 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:32.941843 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.942327 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.943277 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.943465 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.943520 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.945717 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:32.945718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:32.945718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:32.945724 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:32.946915 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.947270 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.947271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.947271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.947319 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.947524 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.947525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.947525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.947621 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:32.947669 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.947897 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.947956 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 23:10:32.948195 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.940013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:42949 (local address 127.15.212.67:34489)
0928 23:10:32.940173 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.940215 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.940747 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.940987 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.940990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.941556 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:32.941748 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.941753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.942406 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.942406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.943191 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.943193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.943542 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.943543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.943645 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.944092 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:32.944100 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:32.945629 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:32.946984 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.946989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.946991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.947205 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.947377 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:32.947378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:32.947378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:32.947471 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:32.947675 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:32.947679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:32.947787 (+ 108us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.947867 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.947983 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260928 23:10:32.948561 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2127
I20260928 23:10:32.948670 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:32.948733 2256 heartbeater.cc:347] Connected to a master server at 127.15.212.126:42949
I20260928 23:10:32.948823 2256 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:32.948861 16209 external_mini_cluster.cc:1428] Setting key 29cc683e1481ec812a5d9fe9fc298eb4
I20260928 23:10:32.949008 2256 heartbeater.cc:511] Master 127.15.212.126:42949 requested a full tablet report, sending...
I20260928 23:10:32.949468 1797 ts_manager.cc:194] Registered new tserver with Master: cb596b38e8294fecb282d35c942dec60 (127.15.212.67:32993)
I20260928 23:10:32.949780 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:32.949918 1797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:34489
I20260928 23:10:32.956987 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.951239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.15.212.126:42949)
0928 23:10:32.951308 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.951310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.951342 (+ 32us) server_negotiation.cc:407] Connection header received
0928 23:10:32.951419 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.951419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.951428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.951474 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:32.951959 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.952396 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.953099 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.953272 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.953323 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.954521 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:32.954522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:32.954522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:32.954528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:32.955864 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.956219 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.956219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.956220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.956267 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:32.956499 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:32.956499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:32.956499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:32.956629 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:32.956711 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.956828 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.956873 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:10:32.958432 1797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:32.960000 1797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:32.966504 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:32993 (local address 127.0.0.1:53094)
0928 23:10:32.962566 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.962576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.962626 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.962963 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.962966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.962968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:32.963035 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.963037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.964079 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.964080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.965338 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.965341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.966194 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.966194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.966276 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.966291 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.966366 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.966414 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":83,"threads_started":1}
I20260928 23:10:32.966532 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.15.212.67:32993)
0928 23:10:32.962771 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.962776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.962787 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:10:32.962832 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.962834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.962852 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.962951 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:32.963168 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.963958 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.965416 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.966109 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.966193 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.966249 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.966394 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.966449 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260928 23:10:32.966977 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:43771 (local address 127.0.0.1:53508)
0928 23:10:32.962443 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.962482 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.962552 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.963014 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.963015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.963018 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:32.963066 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.963068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.964466 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.964467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.965561 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.965564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.966483 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.966484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.966556 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.966816 (+ 260us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.966848 (+ 32us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.966913 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260928 23:10:32.967307 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53508 (local address 127.15.212.66:43771)
0928 23:10:32.962811 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.962815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.962824 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:10:32.962867 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.962870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.962887 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.963147 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:32.963227 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.964330 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.965650 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.966320 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.967084 (+ 764us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.967156 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.967184 (+ 28us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.967228 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":80,"threads_started":1}
I20260928 23:10:32.967360 2190 tablet_service.cc:1467] Processing CreateTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a (DEFAULT_TABLE table=test-table [id=57fe4ab9565c441591e34bf343897105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:32.967743 2190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8203939cfa7e42469ce0c4ac5164d91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:32.967809 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:36739 (local address 127.0.0.1:54286)
0928 23:10:32.962786 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:32.962796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:32.962850 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:32.963099 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:32.963100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:32.963102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:32.963133 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.963134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.965648 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.965648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.966538 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:32.966541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.967570 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.967571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:32.967636 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.967641 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:10:32.967688 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:10:32.967727 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":77,"threads_started":1}
I20260928 23:10:32.967944 2056 tablet_service.cc:1467] Processing CreateTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a (DEFAULT_TABLE table=test-table [id=57fe4ab9565c441591e34bf343897105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:32.968089 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:32.962599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.15.212.65:36739)
0928 23:10:32.962748 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:32.962754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:32.962860 (+ 106us) server_negotiation.cc:407] Connection header received
0928 23:10:32.962920 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:32.962923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:32.962936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:32.963020 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:32.963211 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.964330 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.966636 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:32.967401 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:32.967483 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:32.967536 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:32.967959 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 23:10:32.967999 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260928 23:10:32.968251 2056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8203939cfa7e42469ce0c4ac5164d91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:32.968736 1922 tablet_service.cc:1467] Processing CreateTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a (DEFAULT_TABLE table=test-table [id=57fe4ab9565c441591e34bf343897105]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:32.969048 1922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8203939cfa7e42469ce0c4ac5164d91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:32.971516 2275 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: Bootstrap starting.
I20260928 23:10:32.971728 2274 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Bootstrap starting.
I20260928 23:10:32.971868 2273 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Bootstrap starting.
I20260928 23:10:32.972270 2275 tablet_bootstrap.cc:654] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:32.972302 2274 tablet_bootstrap.cc:654] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:32.972476 2273 tablet_bootstrap.cc:654] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:32.972911 2274 log.cc:824] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:32.972911 2275 log.cc:824] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:32.973057 2273 log.cc:824] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:32.973670 2274 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: No bootstrap required, opened a new log
I20260928 23:10:32.973706 2275 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: No bootstrap required, opened a new log
I20260928 23:10:32.973757 2274 ts_tablet_manager.cc:1397] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:10:32.973762 2275 ts_tablet_manager.cc:1397] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 23:10:32.973795 2273 tablet_bootstrap.cc:492] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: No bootstrap required, opened a new log
I20260928 23:10:32.973861 2273 ts_tablet_manager.cc:1397] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:32.975365 2274 raft_consensus.cc:348] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.975538 2274 raft_consensus.cc:374] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:32.975582 2274 raft_consensus.cc:729] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b158a0e34d54596b07c52beb56bafe9, State: Initialized, Role: FOLLOWER
I20260928 23:10:32.975689 2274 consensus_queue.cc:260] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.975773 2275 raft_consensus.cc:348] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.975782 2273 raft_consensus.cc:348] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.975953 2275 raft_consensus.cc:374] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:32.975975 2273 raft_consensus.cc:374] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:32.975986 2274 ts_tablet_manager.cc:1428] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 23:10:32.976016 2273 raft_consensus.cc:729] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb596b38e8294fecb282d35c942dec60, State: Initialized, Role: FOLLOWER
I20260928 23:10:32.976016 2275 raft_consensus.cc:729] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d03efb01471944ed8f017f32acf59d57, State: Initialized, Role: FOLLOWER
I20260928 23:10:32.976099 2122 heartbeater.cc:503] Master 127.15.212.126:42949 was elected leader, sending a full tablet report...
I20260928 23:10:32.976138 2273 consensus_queue.cc:260] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.976128 2275 consensus_queue.cc:260] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:32.976414 2275 ts_tablet_manager.cc:1428] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 23:10:32.976414 2273 ts_tablet_manager.cc:1428] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:10:32.976487 1988 heartbeater.cc:503] Master 127.15.212.126:42949 was elected leader, sending a full tablet report...
I20260928 23:10:32.976554 2256 heartbeater.cc:503] Master 127.15.212.126:42949 was elected leader, sending a full tablet report...
W20260928 23:10:32.990377 2123 tablet.cc:2369] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:33.009828 2280 raft_consensus.cc:482] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:33.009986 2280 raft_consensus.cc:504] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:33.010412 2280 leader_election.cc:288] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b158a0e34d54596b07c52beb56bafe9 (127.15.212.66:43771), cb596b38e8294fecb282d35c942dec60 (127.15.212.67:32993)
I20260928 23:10:33.014635 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.010827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:48689 (local address 127.15.212.66:43771)
0928 23:10:33.010872 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.010874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.011121 (+ 247us) server_negotiation.cc:407] Connection header received
0928 23:10:33.011202 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.011203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.011217 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.011276 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.011496 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.012353 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.013686 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.014254 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.014337 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.014345 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.014527 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.014566 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:10:33.014635 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.010746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:60371 (local address 127.15.212.67:32993)
0928 23:10:33.010839 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.010841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.010851 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:33.010903 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.010904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.010916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.011062 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.011527 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.012190 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.013659 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.014254 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.014337 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.014345 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.014523 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.014564 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 23:10:33.014665 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.010631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:32993 (local address 127.15.212.65:60371)
0928 23:10:33.010685 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.010697 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.010740 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.011397 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.011398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.011399 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.011455 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.011457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.012280 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.012282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.013547 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.013550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.014349 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.014350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.014436 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.014442 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.014501 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.014547 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":8}
I20260928 23:10:33.014678 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.010815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:43771 (local address 127.15.212.65:48689)
0928 23:10:33.011053 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.011063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.011122 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.011302 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.011303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.011305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.011385 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.011387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.012458 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.012459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.013601 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.013603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.014350 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.014350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.014436 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.014443 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.014501 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.014543 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":39,"thread_start_us":154,"threads_started":1}
I20260928 23:10:33.015101 2210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8203939cfa7e42469ce0c4ac5164d91a" candidate_uuid: "d03efb01471944ed8f017f32acf59d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb596b38e8294fecb282d35c942dec60" is_pre_election: true
I20260928 23:10:33.015273 2210 raft_consensus.cc:2454] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d03efb01471944ed8f017f32acf59d57 in term 0.
I20260928 23:10:33.015600 2076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8203939cfa7e42469ce0c4ac5164d91a" candidate_uuid: "d03efb01471944ed8f017f32acf59d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b158a0e34d54596b07c52beb56bafe9" is_pre_election: true
I20260928 23:10:33.015741 2076 raft_consensus.cc:2454] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d03efb01471944ed8f017f32acf59d57 in term 0.
I20260928 23:10:33.015930 1876 leader_election.cc:302] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb596b38e8294fecb282d35c942dec60, d03efb01471944ed8f017f32acf59d57; no voters:
I20260928 23:10:33.016072 2280 raft_consensus.cc:2781] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:33.016144 2280 raft_consensus.cc:482] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:33.016176 2280 raft_consensus.cc:3037] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.017007 2280 raft_consensus.cc:504] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:33.017162 2280 leader_election.cc:288] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [CANDIDATE]: Term 1 election: Requested vote from peers 2b158a0e34d54596b07c52beb56bafe9 (127.15.212.66:43771), cb596b38e8294fecb282d35c942dec60 (127.15.212.67:32993)
I20260928 23:10:33.017324 2076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8203939cfa7e42469ce0c4ac5164d91a" candidate_uuid: "d03efb01471944ed8f017f32acf59d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b158a0e34d54596b07c52beb56bafe9"
I20260928 23:10:33.017328 2210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8203939cfa7e42469ce0c4ac5164d91a" candidate_uuid: "d03efb01471944ed8f017f32acf59d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb596b38e8294fecb282d35c942dec60"
I20260928 23:10:33.017436 2076 raft_consensus.cc:3037] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.017431 2210 raft_consensus.cc:3037] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.018142 2076 raft_consensus.cc:2454] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d03efb01471944ed8f017f32acf59d57 in term 1.
I20260928 23:10:33.018214 2210 raft_consensus.cc:2454] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d03efb01471944ed8f017f32acf59d57 in term 1.
I20260928 23:10:33.018373 1875 leader_election.cc:302] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b158a0e34d54596b07c52beb56bafe9, d03efb01471944ed8f017f32acf59d57; no voters:
I20260928 23:10:33.018488 2280 raft_consensus.cc:2781] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:33.018680 2280 raft_consensus.cc:686] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [term 1 LEADER]: Becoming Leader. State: Replica: d03efb01471944ed8f017f32acf59d57, State: Running, Role: LEADER
I20260928 23:10:33.018779 2280 consensus_queue.cc:237] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } }
I20260928 23:10:33.019661 1795 catalog_manager.cc:5495] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to d03efb01471944ed8f017f32acf59d57 (127.15.212.65). New cstate: current_term: 1 leader_uuid: "d03efb01471944ed8f017f32acf59d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d03efb01471944ed8f017f32acf59d57" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36739 } health_report { overall_health: HEALTHY } } }
I20260928 23:10:33.026382 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.024095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.15.212.65:36739)
0928 23:10:33.024137 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.024139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.024147 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:10:33.024216 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.024217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.024232 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.024296 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:33.024384 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.024883 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.025665 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.025854 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.025915 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.026051 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:33.026132 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:33.026203 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.026244 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.026253 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:10:33.028038 2210 raft_consensus.cc:1261] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 1 FOLLOWER]: Refusing update from remote peer d03efb01471944ed8f017f32acf59d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:33.028358 2076 raft_consensus.cc:1261] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 1 FOLLOWER]: Refusing update from remote peer d03efb01471944ed8f017f32acf59d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:33.028404 2283 consensus_queue.cc:1035] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb596b38e8294fecb282d35c942dec60" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 32993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:33.028661 2283 consensus_queue.cc:1035] T 8203939cfa7e42469ce0c4ac5164d91a P d03efb01471944ed8f017f32acf59d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b158a0e34d54596b07c52beb56bafe9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:33.031384 2286 mvcc.cc:205] Tried to move back new op lower bound from 7334449287280340992 to 7334449287245598720. Current Snapshot: MvccSnapshot[applied={T|T < 7334449287280340992}]
I20260928 23:10:33.031484 2288 mvcc.cc:205] Tried to move back new op lower bound from 7334449287280340992 to 7334449287245598720. Current Snapshot: MvccSnapshot[applied={T|T < 7334449287280340992 or (T in {7334449287280340992})}]
I20260928 23:10:33.033572 1795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:33.033670 1795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:33.034405 1795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbeb801261b742ad929fed34d752915a: Sending DeleteTablet for 3 replicas of tablet 8203939cfa7e42469ce0c4ac5164d91a
I20260928 23:10:33.034772 1922 tablet_service.cc:1514] Processing DeleteTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:54286
I20260928 23:10:33.034816 2190 tablet_service.cc:1514] Processing DeleteTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53094
I20260928 23:10:33.034812 2056 tablet_service.cc:1514] Processing DeleteTablet for tablet 8203939cfa7e42469ce0c4ac5164d91a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:53508
I20260928 23:10:33.035027 2297 tablet_replica.cc:330] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: stopping tablet replica
I20260928 23:10:33.035130 2297 raft_consensus.cc:2229] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:33.035157 2298 tablet_replica.cc:330] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: stopping tablet replica
I20260928 23:10:33.035235 2297 raft_consensus.cc:2258] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:33.035281 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1859
I20260928 23:10:33.035687 2297 ts_tablet_manager.cc:1905] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:33.035945 2298 raft_consensus.cc:2229] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:33.036111 2298 raft_consensus.cc:2258] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:33.036635 2298 ts_tablet_manager.cc:1905] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:33.037832 2297 ts_tablet_manager.cc:1918] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:33.037914 2297 log.cc:1196] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/8203939cfa7e42469ce0c4ac5164d91a
I20260928 23:10:33.038188 2297 ts_tablet_manager.cc:1939] T 8203939cfa7e42469ce0c4ac5164d91a P cb596b38e8294fecb282d35c942dec60: Deleting consensus metadata
I20260928 23:10:33.038604 2298 ts_tablet_manager.cc:1918] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:33.038640 1782 catalog_manager.cc:4841] TS cb596b38e8294fecb282d35c942dec60 (127.15.212.67:32993): tablet 8203939cfa7e42469ce0c4ac5164d91a (table test-table [id=57fe4ab9565c441591e34bf343897105]) successfully deleted
I20260928 23:10:33.038676 2298 log.cc:1196] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/8203939cfa7e42469ce0c4ac5164d91a
I20260928 23:10:33.038908 2298 ts_tablet_manager.cc:1939] T 8203939cfa7e42469ce0c4ac5164d91a P 2b158a0e34d54596b07c52beb56bafe9: Deleting consensus metadata
I20260928 23:10:33.039350 1781 catalog_manager.cc:4841] TS 2b158a0e34d54596b07c52beb56bafe9 (127.15.212.66:43771): tablet 8203939cfa7e42469ce0c4ac5164d91a (table test-table [id=57fe4ab9565c441591e34bf343897105]) successfully deleted
W20260928 23:10:33.041491 1781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:36739 (error 108)
I20260928 23:10:33.041711 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1993
I20260928 23:10:33.042428 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.042310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:36739 (local address 127.0.0.1:54314)
0928 23:10:33.042364 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.042400 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:36739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 23:10:33.042582 1781 catalog_manager.cc:4584] TS d03efb01471944ed8f017f32acf59d57 (127.15.212.65:36739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8203939cfa7e42469ce0c4ac5164d91a: Network error: Client connection negotiation failed: client connection to 127.15.212.65:36739: connect: Connection refused (error 111)
I20260928 23:10:33.047397 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2127
I20260928 23:10:33.051524 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 1765
2026-09-28T23:10:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1942 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T23:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:33Z Disabled control of system clock
I20260928 23:10:33.078459 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40987
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:40987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:33.158300 2303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:33.158507 2303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:33.158527 2303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:33.159685 2303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:33.159727 2303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:33.159740 2303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:33.159752 2303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:33.159765 2303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:33.161156 2303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:40987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:33.161423 2303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:33.161712 2303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:33.166496 2311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.166500 2309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.166584 2308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:33.166920 2303 server_base.cc:1034] running on GCE node
I20260928 23:10:33.167174 2303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:33.167457 2303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:33.168591 2303 hybrid_clock.cc:648] HybridClock initialized: now 1790637033168560 us; error 46 us; skew 500 ppm
I20260928 23:10:33.169823 2303 webserver.cc:460] Webserver started at http://127.15.212.126:40943/ using document root <none> and password file <none>
I20260928 23:10:33.170089 2303 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:33.170158 2303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:33.170292 2303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:33.171329 2303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "0b953e8faa134c3484f90fdaa72742ab"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "23332bb93568bac8cf7f06d3e325b135"
server_key_iv: "353714fddfc6e2c0438bb1378727fc18"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.171689 2303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "0b953e8faa134c3484f90fdaa72742ab"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "23332bb93568bac8cf7f06d3e325b135"
server_key_iv: "353714fddfc6e2c0438bb1378727fc18"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.173127 2303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:33.173920 2317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.174096 2303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:33.174202 2303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "0b953e8faa134c3484f90fdaa72742ab"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "23332bb93568bac8cf7f06d3e325b135"
server_key_iv: "353714fddfc6e2c0438bb1378727fc18"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.174284 2303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:33.188274 2303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:33.188664 2303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:33.188831 2303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:33.192876 2303 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:40987
I20260928 23:10:33.192922 2369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:40987 every 8 connection(s)
I20260928 23:10:33.193295 2303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:33.194233 2370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:33.196499 2370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: Bootstrap starting.
I20260928 23:10:33.197093 2370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:33.197329 2370 log.cc:824] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:33.197983 2370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: No bootstrap required, opened a new log
I20260928 23:10:33.199514 2370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } }
I20260928 23:10:33.199648 2370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:33.199671 2370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b953e8faa134c3484f90fdaa72742ab, State: Initialized, Role: FOLLOWER
I20260928 23:10:33.199813 2370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } }
I20260928 23:10:33.199893 2370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:33.199927 2370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:33.199958 2370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.200428 2370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } }
I20260928 23:10:33.200531 2370 leader_election.cc:302] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b953e8faa134c3484f90fdaa72742ab; no voters:
I20260928 23:10:33.200649 2370 leader_election.cc:288] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:33.200724 2373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:33.200909 2373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [term 1 LEADER]: Becoming Leader. State: Replica: 0b953e8faa134c3484f90fdaa72742ab, State: Running, Role: LEADER
I20260928 23:10:33.200915 2370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:33.201025 2373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } }
I20260928 23:10:33.201540 2374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b953e8faa134c3484f90fdaa72742ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } } }
I20260928 23:10:33.201644 2374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:33.201817 2375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b953e8faa134c3484f90fdaa72742ab. Latest consensus state: current_term: 1 leader_uuid: "0b953e8faa134c3484f90fdaa72742ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b953e8faa134c3484f90fdaa72742ab" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40987 } } }
I20260928 23:10:33.201849 2382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:33.201875 2375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:33.202374 2382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:33.202800 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2303
I20260928 23:10:33.202912 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:33.203120 16209 external_mini_cluster.cc:1428] Setting key 091901931f4290e2e5552cf9c90f9b1f
I20260928 23:10:33.204895 2382 catalog_manager.cc:1347] Generated new cluster ID: 2c9d37ec2423424e911e73241db0469c
I20260928 23:10:33.204952 2382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:33.208547 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.203546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.15.212.126:40987)
0928 23:10:33.203964 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.203973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.204007 (+ 34us) server_negotiation.cc:407] Connection header received
0928 23:10:33.204119 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.204125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.204194 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.204341 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:33.204715 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.205710 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.206517 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.206971 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.207105 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.207810 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:33.207814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:33.207814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:33.208005 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:33.208038 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:33.208048 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:33.208128 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.208205 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.208286 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":81,"threads_started":1}
I20260928 23:10:33.221438 2382 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:33.222025 2382 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:33.224649 2382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: Generated new TSK 0
I20260928 23:10:33.224856 2382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:33.231561 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--builtin_ntp_servers=127.15.212.84:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:33.334892 2394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:33.335136 2394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:33.335175 2394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:33.336352 2394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:33.336421 2394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:33.336481 2394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:33.337864 2394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:33.338122 2394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:33.338430 2394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:33.343433 2399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.343472 2400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:33.343570 2394 server_base.cc:1034] running on GCE node
W20260928 23:10:33.343698 2402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:33.343917 2394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:33.344183 2394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:33.345351 2394 hybrid_clock.cc:648] HybridClock initialized: now 1790637033345332 us; error 31 us; skew 500 ppm
I20260928 23:10:33.350366 2394 webserver.cc:460] Webserver started at http://127.15.212.65:40403/ using document root <none> and password file <none>
I20260928 23:10:33.350723 2394 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:33.350808 2394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:33.351027 2394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:33.352514 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "dfe5587b8eae408593c1edc832823fd0"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "38e285447f98f6394569928823269ced"
server_key_iv: "5c8d980c86e3918140c64b7f35190488"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.352913 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "dfe5587b8eae408593c1edc832823fd0"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "38e285447f98f6394569928823269ced"
server_key_iv: "5c8d980c86e3918140c64b7f35190488"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.354533 2394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:10:33.355453 2408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.355686 2394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:10:33.355801 2394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "dfe5587b8eae408593c1edc832823fd0"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "38e285447f98f6394569928823269ced"
server_key_iv: "5c8d980c86e3918140c64b7f35190488"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.355899 2394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:33.376436 2394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:33.376842 2394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:33.377084 2394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:33.377362 2394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:33.377753 2394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:33.377789 2394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.377836 2394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:33.377866 2394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.384038 2394 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:36501
I20260928 23:10:33.384122 2521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:36501 every 8 connection(s)
I20260928 23:10:33.384570 2394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:33.389700 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.385420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:54897 (local address 127.15.212.126:40987)
0928 23:10:33.385472 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.385474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.385692 (+ 218us) server_negotiation.cc:407] Connection header received
0928 23:10:33.386175 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.386176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.386187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.386252 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:33.386924 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.387412 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.388364 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.388626 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.388682 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.389458 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:33.389459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:33.389460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:33.389464 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:33.389478 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:33.389482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:33.389534 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.389609 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.389637 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:10:33.389935 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2394
I20260928 23:10:33.389955 2524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.385343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40987 (local address 127.15.212.65:54897)
0928 23:10:33.385549 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.385593 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.386091 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.386334 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.386338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.386346 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:33.386852 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.386855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.387493 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.387493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.388288 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.388290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.388702 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.388702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.388821 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.389327 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:33.389327 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:33.389355 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:33.389378 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:33.389537 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:33.389541 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:33.389544 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.389597 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.389681 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":128,"threads_started":1}
I20260928 23:10:33.390043 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:33.390275 16209 external_mini_cluster.cc:1428] Setting key 12c8af6e55b2dc136f43b8a2090cb6c7
I20260928 23:10:33.390545 2522 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40987
I20260928 23:10:33.390658 2522 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:33.390848 2522 heartbeater.cc:511] Master 127.15.212.126:40987 requested a full tablet report, sending...
I20260928 23:10:33.391206 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--builtin_ntp_servers=127.15.212.84:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 23:10:33.391438 2334 ts_manager.cc:194] Registered new tserver with Master: dfe5587b8eae408593c1edc832823fd0 (127.15.212.65:36501)
I20260928 23:10:33.392041 2334 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.65:54897
W20260928 23:10:33.473300 2525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:33.473518 2525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:33.473541 2525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:33.474630 2525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:33.474665 2525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:33.474689 2525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:33.476044 2525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:33.476236 2525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:33.476550 2525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:33.481531 2533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.481547 2530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.481575 2531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:33.481797 2525 server_base.cc:1034] running on GCE node
I20260928 23:10:33.482044 2525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:33.482311 2525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:33.483476 2525 hybrid_clock.cc:648] HybridClock initialized: now 1790637033483464 us; error 24 us; skew 500 ppm
I20260928 23:10:33.484773 2525 webserver.cc:460] Webserver started at http://127.15.212.66:44523/ using document root <none> and password file <none>
I20260928 23:10:33.485021 2525 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:33.485090 2525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:33.485245 2525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:33.486304 2525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "acc987c4367446dabe71e0257cbf568e"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "5541bb320199a7224617ca07d1fa0548"
server_key_iv: "41de511a33563228b182d11def97d971"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.486665 2525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "acc987c4367446dabe71e0257cbf568e"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "5541bb320199a7224617ca07d1fa0548"
server_key_iv: "41de511a33563228b182d11def97d971"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.487924 2525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:33.488672 2539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.488828 2525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:33.488930 2525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "acc987c4367446dabe71e0257cbf568e"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "5541bb320199a7224617ca07d1fa0548"
server_key_iv: "41de511a33563228b182d11def97d971"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.489001 2525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:33.501854 2525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:33.502209 2525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:33.502367 2525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:33.502596 2525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:33.502985 2525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:33.503047 2525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.503087 2525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:33.503115 2525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.508685 2525 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:33641
I20260928 23:10:33.508725 2652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:33641 every 8 connection(s)
I20260928 23:10:33.509099 2525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:10:33.514382 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.509801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:39101 (local address 127.15.212.126:40987)
0928 23:10:33.509851 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.509853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.510105 (+ 252us) server_negotiation.cc:407] Connection header received
0928 23:10:33.510677 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.510679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.510692 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.510738 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:33.511430 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.511925 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.512996 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.513201 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.513275 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.514077 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:33.514079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:33.514079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:33.514084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:33.514099 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:33.514103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:33.514169 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.514274 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.514300 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:10:33.514611 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.509785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40987 (local address 127.15.212.66:39101)
0928 23:10:33.509968 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.510008 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.510554 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.510827 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.510836 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.510844 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:33.511331 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.511335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.512027 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.512028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.512903 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.512905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.513296 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.513297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.513424 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.513924 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:33.513924 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:33.513951 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:33.513973 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:33.514171 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:33.514177 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:33.514181 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.514263 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.514364 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260928 23:10:33.515082 2653 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40987
I20260928 23:10:33.515183 2653 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:33.515372 2653 heartbeater.cc:511] Master 127.15.212.126:40987 requested a full tablet report, sending...
I20260928 23:10:33.515725 2334 ts_manager.cc:194] Registered new tserver with Master: acc987c4367446dabe71e0257cbf568e (127.15.212.66:33641)
I20260928 23:10:33.516062 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2525
I20260928 23:10:33.516152 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:33.516274 2334 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.66:39101
I20260928 23:10:33.516373 16209 external_mini_cluster.cc:1428] Setting key 7f6b91182bb38d086c3de02dfbd02f62
I20260928 23:10:33.517365 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--builtin_ntp_servers=127.15.212.84:36223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:33.597554 2656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:33.597750 2656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:33.597771 2656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:33.598846 2656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:33.598881 2656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:33.598904 2656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:33.600143 2656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:36223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:40987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:33.600382 2656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:33.600698 2656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:33.605823 2664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.605845 2662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:33.605823 2661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:33.605903 2656 server_base.cc:1034] running on GCE node
I20260928 23:10:33.606233 2656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:33.606498 2656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:33.607683 2656 hybrid_clock.cc:648] HybridClock initialized: now 1790637033607665 us; error 33 us; skew 500 ppm
I20260928 23:10:33.609098 2656 webserver.cc:460] Webserver started at http://127.15.212.67:36519/ using document root <none> and password file <none>
I20260928 23:10:33.609360 2656 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:33.609449 2656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:33.609597 2656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:33.610649 2656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "f1971639ac4141cbafd1f4cb2ac414d5"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "1878aadeb1845f7926188b036d61053d"
server_key_iv: "1d17e598d1168b607bf206d9212eb3ba"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.611001 2656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "f1971639ac4141cbafd1f4cb2ac414d5"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "1878aadeb1845f7926188b036d61053d"
server_key_iv: "1d17e598d1168b607bf206d9212eb3ba"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.612344 2656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:10:33.613087 2670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.613250 2656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:33.613349 2656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "f1971639ac4141cbafd1f4cb2ac414d5"
format_stamp: "Formatted at 2026-09-28 23:10:33 on dist-test-slave-x2t8"
server_key: "1878aadeb1845f7926188b036d61053d"
server_key_iv: "1d17e598d1168b607bf206d9212eb3ba"
server_key_version: "encryptionkey@0"
I20260928 23:10:33.613427 2656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:33.628499 2656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:33.628837 2656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:33.628994 2656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:33.629233 2656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:33.629575 2656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:33.629627 2656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.629670 2656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:33.629693 2656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:33.635437 2656 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:33847
I20260928 23:10:33.635483 2783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:33847 every 8 connection(s)
I20260928 23:10:33.635843 2656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:10:33.640829 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.636647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:54883 (local address 127.15.212.126:40987)
0928 23:10:33.636712 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.636714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.636920 (+ 206us) server_negotiation.cc:407] Connection header received
0928 23:10:33.637403 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.637404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.637412 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.637474 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:33.638134 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.638617 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.639544 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.639724 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.639774 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.640612 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:33.640613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:33.640613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:33.640617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:33.640630 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:33.640634 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:33.640678 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.640746 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.640770 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 23:10:33.641083 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.636577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:40987 (local address 127.15.212.67:54883)
0928 23:10:33.636783 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.636823 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.637302 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.637532 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.637535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.637544 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:33.638058 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.638062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.638697 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.638698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.639469 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.639471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.639797 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.639798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.639915 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.640414 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:33.640415 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:33.640499 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:33.640525 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:33.640690 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:33.640694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:33.640697 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.640747 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.640825 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260928 23:10:33.641624 2784 heartbeater.cc:347] Connected to a master server at 127.15.212.126:40987
I20260928 23:10:33.641736 2784 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:33.641918 2784 heartbeater.cc:511] Master 127.15.212.126:40987 requested a full tablet report, sending...
I20260928 23:10:33.642148 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2656
I20260928 23:10:33.642248 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:33.642342 2334 ts_manager.cc:194] Registered new tserver with Master: f1971639ac4141cbafd1f4cb2ac414d5 (127.15.212.67:33847)
I20260928 23:10:33.642474 16209 external_mini_cluster.cc:1428] Setting key 325280f49bae75530c32a129474b2f17
I20260928 23:10:33.642792 2334 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.67:54883
I20260928 23:10:33.643321 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:33.647691 2392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.644740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.15.212.126:40987)
0928 23:10:33.644779 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.644781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.644935 (+ 154us) server_negotiation.cc:407] Connection header received
0928 23:10:33.645009 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.645010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.645022 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.645076 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:33.645243 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.645711 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.646479 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.646668 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.646750 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.647393 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:33.647394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:33.647395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:33.647399 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:33.647411 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:33.647416 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:33.647519 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.647585 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.647594 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:33.649118 2334 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:33.650604 2334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:33.657104 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.653054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33847 (local address 127.0.0.1:50006)
0928 23:10:33.653430 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.653440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.653491 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.653739 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.653740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.653742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.653787 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.653788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.654623 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.654624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.655880 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.655883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.656804 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.656892 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.656908 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.656973 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.657027 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":61,"threads_started":1}
I20260928 23:10:33.657229 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.652949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.15.212.65:36501)
0928 23:10:33.653379 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.653383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.653393 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:33.653429 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.653431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.653445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.653715 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.653776 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.654852 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656102 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.656831 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.657007 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.657104 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.657140 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":91,"threads_started":1}
I20260928 23:10:33.657478 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.653164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.15.212.67:33847)
0928 23:10:33.653357 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.653361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.653547 (+ 186us) server_negotiation.cc:407] Connection header received
0928 23:10:33.653589 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.653591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.653608 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.653678 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.653859 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.654538 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.655960 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.656706 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656779 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.656874 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.657358 (+ 484us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.657401 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260928 23:10:33.657562 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.652833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:36501 (local address 127.0.0.1:51568)
0928 23:10:33.652985 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.653032 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.653116 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.653555 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.653557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.653562 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.653635 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.653638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.654958 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.654959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.656025 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.656027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.656933 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.656996 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.657000 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.657088 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.657131 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":9,"thread_start_us":71,"threads_started":1}
I20260928 23:10:33.658126 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.652839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:33641 (local address 127.0.0.1:40342)
0928 23:10:33.653166 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.653176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.653228 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.653554 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.653557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.653559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.653635 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.653637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.656090 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.656837 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.656838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.657904 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.657905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.657965 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.657968 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.658026 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.658062 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260928 23:10:33.658367 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.653146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.15.212.66:33641)
0928 23:10:33.653308 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.653313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.653324 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:10:33.653366 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.653368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.653385 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.653471 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.653719 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.654847 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.656927 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.657696 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.657765 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.657843 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.658066 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.658109 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260928 23:10:33.658303 2456 tablet_service.cc:1467] Processing CreateTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 (DEFAULT_TABLE table=test-table [id=7422f819f4e64ed0aa45e803f7e746a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:33.658311 2718 tablet_service.cc:1467] Processing CreateTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 (DEFAULT_TABLE table=test-table [id=7422f819f4e64ed0aa45e803f7e746a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:33.658641 2456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f8ec1c4fa542fdb582861c97b04629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:33.658656 2718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f8ec1c4fa542fdb582861c97b04629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:33.659175 2587 tablet_service.cc:1467] Processing CreateTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 (DEFAULT_TABLE table=test-table [id=7422f819f4e64ed0aa45e803f7e746a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:33.659480 2587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05f8ec1c4fa542fdb582861c97b04629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:33.661630 2798 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Bootstrap starting.
I20260928 23:10:33.662225 2799 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Bootstrap starting.
I20260928 23:10:33.662380 2798 tablet_bootstrap.cc:654] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:33.662729 2800 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Bootstrap starting.
I20260928 23:10:33.662839 2798 log.cc:824] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:33.663009 2799 tablet_bootstrap.cc:654] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:33.663331 2799 log.cc:824] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:33.663432 2800 tablet_bootstrap.cc:654] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:33.663671 2798 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: No bootstrap required, opened a new log
I20260928 23:10:33.663743 2798 ts_tablet_manager.cc:1397] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:33.663736 2800 log.cc:824] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:33.664024 2799 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: No bootstrap required, opened a new log
I20260928 23:10:33.664083 2799 ts_tablet_manager.cc:1397] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:33.664587 2800 tablet_bootstrap.cc:492] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: No bootstrap required, opened a new log
I20260928 23:10:33.664638 2800 ts_tablet_manager.cc:1397] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:33.666126 2799 raft_consensus.cc:348] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.666241 2799 raft_consensus.cc:374] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:33.666261 2799 raft_consensus.cc:729] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1971639ac4141cbafd1f4cb2ac414d5, State: Initialized, Role: FOLLOWER
I20260928 23:10:33.666383 2799 consensus_queue.cc:260] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.666393 2798 raft_consensus.cc:348] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.666558 2798 raft_consensus.cc:374] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:33.666595 2798 raft_consensus.cc:729] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe5587b8eae408593c1edc832823fd0, State: Initialized, Role: FOLLOWER
I20260928 23:10:33.666627 2799 ts_tablet_manager.cc:1428] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:10:33.666656 2784 heartbeater.cc:503] Master 127.15.212.126:40987 was elected leader, sending a full tablet report...
I20260928 23:10:33.666705 2798 consensus_queue.cc:260] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.666951 2798 ts_tablet_manager.cc:1428] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 23:10:33.667027 2522 heartbeater.cc:503] Master 127.15.212.126:40987 was elected leader, sending a full tablet report...
I20260928 23:10:33.668083 2800 raft_consensus.cc:348] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.668226 2800 raft_consensus.cc:374] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:33.668269 2800 raft_consensus.cc:729] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc987c4367446dabe71e0257cbf568e, State: Initialized, Role: FOLLOWER
I20260928 23:10:33.668373 2800 consensus_queue.cc:260] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.668639 2800 ts_tablet_manager.cc:1428] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260928 23:10:33.668715 2653 heartbeater.cc:503] Master 127.15.212.126:40987 was elected leader, sending a full tablet report...
W20260928 23:10:33.759675 2654 tablet.cc:2369] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:33.855346 2806 raft_consensus.cc:482] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:33.855530 2806 raft_consensus.cc:504] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.855947 2806 leader_election.cc:288] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfe5587b8eae408593c1edc832823fd0 (127.15.212.65:36501), f1971639ac4141cbafd1f4cb2ac414d5 (127.15.212.67:33847)
I20260928 23:10:33.858814 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.856130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:36501 (local address 127.15.212.66:59703)
0928 23:10:33.856192 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.856207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.856268 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.856528 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.856530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.856532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.856614 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.856616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.857254 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.857255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.858013 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.858015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.858551 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.858632 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.858636 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.858698 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.858742 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:10:33.858812 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.856219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:59703 (local address 127.15.212.65:36501)
0928 23:10:33.856259 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.856261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.856270 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:10:33.856381 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.856382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.856396 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.856521 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.856694 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.857170 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858116 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.858456 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858570 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.858579 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.858715 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.858748 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:10:33.859092 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.856343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:37219 (local address 127.15.212.67:33847)
0928 23:10:33.856414 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.856417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.856465 (+ 48us) server_negotiation.cc:407] Connection header received
0928 23:10:33.856575 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.856577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.856592 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.856667 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:33.856819 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.857269 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858209 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.858664 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858755 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.858764 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.858961 (+ 197us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.859023 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 23:10:33.859156 2476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f8ec1c4fa542fdb582861c97b04629" candidate_uuid: "acc987c4367446dabe71e0257cbf568e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe5587b8eae408593c1edc832823fd0" is_pre_election: true
I20260928 23:10:33.859328 2476 raft_consensus.cc:2454] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc987c4367446dabe71e0257cbf568e in term 0.
I20260928 23:10:33.859339 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.856250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:33847 (local address 127.15.212.66:37219)
0928 23:10:33.856424 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.856433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:33.856495 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:33.856674 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:33.856674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:33.856676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:33.856749 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.856751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.857365 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.857366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.858109 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:33.858111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.858764 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.858765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:33.858846 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.858849 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:33.859190 (+ 341us) client_negotiation.cc:240] Negotiation successful
0928 23:10:33.859254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
I20260928 23:10:33.859603 2540 leader_election.cc:302] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acc987c4367446dabe71e0257cbf568e, dfe5587b8eae408593c1edc832823fd0; no voters:
I20260928 23:10:33.859717 2738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f8ec1c4fa542fdb582861c97b04629" candidate_uuid: "acc987c4367446dabe71e0257cbf568e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" is_pre_election: true
I20260928 23:10:33.859769 2806 raft_consensus.cc:2781] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:33.859838 2806 raft_consensus.cc:482] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:33.859863 2806 raft_consensus.cc:3037] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.859889 2738 raft_consensus.cc:2454] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acc987c4367446dabe71e0257cbf568e in term 0.
I20260928 23:10:33.860797 2806 raft_consensus.cc:504] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.860937 2806 leader_election.cc:288] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [CANDIDATE]: Term 1 election: Requested vote from peers dfe5587b8eae408593c1edc832823fd0 (127.15.212.65:36501), f1971639ac4141cbafd1f4cb2ac414d5 (127.15.212.67:33847)
I20260928 23:10:33.861080 2476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f8ec1c4fa542fdb582861c97b04629" candidate_uuid: "acc987c4367446dabe71e0257cbf568e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe5587b8eae408593c1edc832823fd0"
I20260928 23:10:33.861142 2738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05f8ec1c4fa542fdb582861c97b04629" candidate_uuid: "acc987c4367446dabe71e0257cbf568e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1971639ac4141cbafd1f4cb2ac414d5"
I20260928 23:10:33.861163 2476 raft_consensus.cc:3037] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.861227 2738 raft_consensus.cc:3037] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:33.861886 2476 raft_consensus.cc:2454] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc987c4367446dabe71e0257cbf568e in term 1.
I20260928 23:10:33.861985 2738 raft_consensus.cc:2454] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acc987c4367446dabe71e0257cbf568e in term 1.
I20260928 23:10:33.862068 2540 leader_election.cc:302] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acc987c4367446dabe71e0257cbf568e, dfe5587b8eae408593c1edc832823fd0; no voters:
I20260928 23:10:33.862187 2806 raft_consensus.cc:2781] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:33.862358 2806 raft_consensus.cc:686] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 1 LEADER]: Becoming Leader. State: Replica: acc987c4367446dabe71e0257cbf568e, State: Running, Role: LEADER
I20260928 23:10:33.862457 2806 consensus_queue.cc:237] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } }
I20260928 23:10:33.863283 2334 catalog_manager.cc:5495] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e reported cstate change: term changed from 0 to 1, leader changed from <none> to acc987c4367446dabe71e0257cbf568e (127.15.212.66). New cstate: current_term: 1 leader_uuid: "acc987c4367446dabe71e0257cbf568e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acc987c4367446dabe71e0257cbf568e" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 33641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 } health_report { overall_health: UNKNOWN } } }
W20260928 23:10:33.885262 2523 tablet.cc:2369] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:33.886368 2785 tablet.cc:2369] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:33.887951 2796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.885777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.15.212.66:33641)
0928 23:10:33.885835 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:33.885837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:33.885846 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:10:33.885900 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:33.885900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:33.885919 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:33.885981 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:33.886104 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.886611 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.887325 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:33.887501 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:33.887549 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:33.887679 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:33.887761 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:33.887828 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:33.887871 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 23:10:33.887878 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:10:33.889786 2738 raft_consensus.cc:1261] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 1 FOLLOWER]: Refusing update from remote peer acc987c4367446dabe71e0257cbf568e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:33.889832 2476 raft_consensus.cc:1261] T 05f8ec1c4fa542fdb582861c97b04629 P dfe5587b8eae408593c1edc832823fd0 [term 1 FOLLOWER]: Refusing update from remote peer acc987c4367446dabe71e0257cbf568e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:33.890179 2806 consensus_queue.cc:1035] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1971639ac4141cbafd1f4cb2ac414d5" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 33847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:33.890348 2808 consensus_queue.cc:1035] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe5587b8eae408593c1edc832823fd0" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 36501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:33.892830 2813 mvcc.cc:205] Tried to move back new op lower bound from 7334449290809810944 to 7334449290701119488. Current Snapshot: MvccSnapshot[applied={T|T < 7334449290809810944}]
I20260928 23:10:33.895538 2334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:33.895643 2334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:33.896373 2334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b953e8faa134c3484f90fdaa72742ab: Sending DeleteTablet for 3 replicas of tablet 05f8ec1c4fa542fdb582861c97b04629
I20260928 23:10:33.896690 2587 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='slave'} at 127.0.0.1:40342
I20260928 23:10:33.896749 2456 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='slave'} at 127.0.0.1:51568
I20260928 23:10:33.896916 2821 tablet_replica.cc:330] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: stopping tablet replica
I20260928 23:10:33.896944 2718 tablet_service.cc:1514] Processing DeleteTablet for tablet 05f8ec1c4fa542fdb582861c97b04629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:33 UTC) from {username='slave'} at 127.0.0.1:50006
I20260928 23:10:33.897058 2821 raft_consensus.cc:2229] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:10:33.897111 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2394
I20260928 23:10:33.897689 2823 tablet_replica.cc:330] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: stopping tablet replica
I20260928 23:10:33.897791 2823 raft_consensus.cc:2229] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:33.897791 2821 raft_consensus.cc:2258] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:33.897899 2823 raft_consensus.cc:2258] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:33.898339 2821 ts_tablet_manager.cc:1905] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:33.900008 2821 ts_tablet_manager.cc:1918] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:33.900094 2821 log.cc:1196] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/05f8ec1c4fa542fdb582861c97b04629
I20260928 23:10:33.900346 2821 ts_tablet_manager.cc:1939] T 05f8ec1c4fa542fdb582861c97b04629 P acc987c4367446dabe71e0257cbf568e: Deleting consensus metadata
I20260928 23:10:33.901021 2321 catalog_manager.cc:4841] TS acc987c4367446dabe71e0257cbf568e (127.15.212.66:33641): tablet 05f8ec1c4fa542fdb582861c97b04629 (table test-table [id=7422f819f4e64ed0aa45e803f7e746a2]) successfully deleted
W20260928 23:10:33.902804 2318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:36501 (error 108)
I20260928 23:10:33.903353 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:33.903201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:36501 (local address 127.0.0.1:51570)
0928 23:10:33.903272 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:33.903314 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:36501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 23:10:33.903474 2318 catalog_manager.cc:4584] TS dfe5587b8eae408593c1edc832823fd0 (127.15.212.65:36501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05f8ec1c4fa542fdb582861c97b04629: Network error: Client connection negotiation failed: client connection to 127.15.212.65:36501: connect: Connection refused (error 111)
I20260928 23:10:33.903600 2823 ts_tablet_manager.cc:1905] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:33.904683 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2525
I20260928 23:10:33.905385 2823 ts_tablet_manager.cc:1918] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:33.905471 2823 log.cc:1196] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/05f8ec1c4fa542fdb582861c97b04629
I20260928 23:10:33.905730 2823 ts_tablet_manager.cc:1939] T 05f8ec1c4fa542fdb582861c97b04629 P f1971639ac4141cbafd1f4cb2ac414d5: Deleting consensus metadata
I20260928 23:10:33.906140 2318 catalog_manager.cc:4841] TS f1971639ac4141cbafd1f4cb2ac414d5 (127.15.212.67:33847): tablet 05f8ec1c4fa542fdb582861c97b04629 (table test-table [id=7422f819f4e64ed0aa45e803f7e746a2]) successfully deleted
I20260928 23:10:33.909972 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2656
I20260928 23:10:33.914377 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2303
2026-09-28T23:10:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (862 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:33 dist-test-slave-x2t8 krb5kdc[2827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:33 dist-test-slave-x2t8 krb5kdc[2827](info): set up 2 sockets
Sep 28 23:10:33 dist-test-slave-x2t8 krb5kdc[2827](info): commencing operation
krb5kdc: starting...
W20260928 23:10:34.940111 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:35.051333 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36817
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:36817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:35.129611 2840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:35.129835 2840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:35.129873 2840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:35.130939 2840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:35.130972 2840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:35.130986 2840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:35.130999 2840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:35.131011 2840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:35.132350 2840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:36817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:36817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:35.132601 2840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:35.132850 2840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:35.137501 2845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.137494 2848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.137662 2840 server_base.cc:1034] running on GCE node
W20260928 23:10:35.137773 2846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.138005 2840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:35.138309 2840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:35.139482 2840 hybrid_clock.cc:648] HybridClock initialized: now 1790637035139467 us; error 31 us; skew 500 ppm
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:35.141803 2840 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:35.142196 2840 webserver.cc:460] Webserver started at http://127.15.212.126:44137/ using document root <none> and password file <none>
I20260928 23:10:35.142414 2840 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:35.142488 2840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:35.142630 2840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:35.143616 2840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "c55b9537ddbc43a8a550797746ccb4ed"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "fd789cf0d97ac96876c2cdde2884389f"
server_key_iv: "eaca6a889a238154941108e88dfc1870"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.143958 2840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "c55b9537ddbc43a8a550797746ccb4ed"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "fd789cf0d97ac96876c2cdde2884389f"
server_key_iv: "eaca6a889a238154941108e88dfc1870"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.145473 2840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:35.146180 2855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.146328 2840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:35.146449 2840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "c55b9537ddbc43a8a550797746ccb4ed"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "fd789cf0d97ac96876c2cdde2884389f"
server_key_iv: "eaca6a889a238154941108e88dfc1870"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.146525 2840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:35.167362 2840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:35.169210 2840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:35.169384 2840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:35.173019 2840 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:36817
I20260928 23:10:35.173071 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:36817 every 8 connection(s)
I20260928 23:10:35.173440 2840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:35.174281 2908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:35.175539 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2840
I20260928 23:10:35.175660 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:35.175846 16209 external_mini_cluster.cc:1428] Setting key d752b6daf350e3425ce8e7f402ae12b5
I20260928 23:10:35.176867 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: Bootstrap starting.
I20260928 23:10:35.177565 2908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:35.177896 2908 log.cc:824] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:35.178580 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: No bootstrap required, opened a new log
I20260928 23:10:35.180974 2908 raft_consensus.cc:348] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } }
I20260928 23:10:35.181138 2908 raft_consensus.cc:374] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:35.181180 2908 raft_consensus.cc:729] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55b9537ddbc43a8a550797746ccb4ed, State: Initialized, Role: FOLLOWER
I20260928 23:10:35.181310 2908 consensus_queue.cc:260] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } }
I20260928 23:10:35.181393 2908 raft_consensus.cc:388] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:35.181427 2908 raft_consensus.cc:482] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:35.181483 2908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:35.182150 2908 raft_consensus.cc:504] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } }
I20260928 23:10:35.182236 2908 leader_election.cc:302] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c55b9537ddbc43a8a550797746ccb4ed; no voters:
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:35.182436 2908 leader_election.cc:288] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:35.182459 2913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:35.182564 2913 raft_consensus.cc:686] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [term 1 LEADER]: Becoming Leader. State: Replica: c55b9537ddbc43a8a550797746ccb4ed, State: Running, Role: LEADER
I20260928 23:10:35.182650 2913 consensus_queue.cc:237] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } }
I20260928 23:10:35.182703 2908 sys_catalog.cc:564] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:35.182971 2915 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c55b9537ddbc43a8a550797746ccb4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } } }
I20260928 23:10:35.183061 2915 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:35.183280 2918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:35.183543 2914 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c55b9537ddbc43a8a550797746ccb4ed. Latest consensus state: current_term: 1 leader_uuid: "c55b9537ddbc43a8a550797746ccb4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55b9537ddbc43a8a550797746ccb4ed" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 36817 } } }
I20260928 23:10:35.183624 2914 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:35.183933 2918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:35.185808 2918 catalog_manager.cc:1347] Generated new cluster ID: 213d98ee24184176be960eac28bd1659
I20260928 23:10:35.185856 2918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:35.186523 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.176302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.15.212.126:36817)
0928 23:10:35.176483 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.176490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.176513 (+ 23us) server_negotiation.cc:407] Connection header received
0928 23:10:35.176882 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.176887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.176956 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.177070 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:35.177889 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.178831 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.179814 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.180183 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.180254 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.182987 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:35.182991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:35.182991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:35.183003 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:35.184997 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.185357 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.185360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.185360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.185458 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.185680 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.185681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.185681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.185943 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:35.186099 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.186211 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.186338 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260928 23:10:35.201676 2918 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:35.202154 2918 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:35.206839 2918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: Generated new TSK 0
I20260928 23:10:35.207020 2918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.212.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
Entry for principal kudu/127.15.212.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab.
I20260928 23:10:35.237347 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:35.315248 2934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:35.315476 2934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:35.315515 2934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:35.316673 2934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:35.316735 2934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:35.316779 2934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:35.318220 2934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.65.keytab
--principal=kudu/127.15.212.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:35.318477 2934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:35.318706 2934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:35.323414 2942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.323494 2940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.323414 2939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.323619 2934 server_base.cc:1034] running on GCE node
I20260928 23:10:35.323804 2934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:35.324041 2934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:35.325202 2934 hybrid_clock.cc:648] HybridClock initialized: now 1790637035325187 us; error 46 us; skew 500 ppm
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:35.327561 2934 init.cc:377] Logged in from keytab as kudu/127.15.212.65@KRBTEST.COM (short username kudu)
I20260928 23:10:35.327965 2934 webserver.cc:460] Webserver started at http://127.15.212.65:36209/ using document root <none> and password file <none>
I20260928 23:10:35.328200 2934 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:35.328269 2934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:35.328428 2934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:35.329406 2934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "e26b7ea2be3e489aad171fe1d2eb7cac"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "d0d66aa8566a19e28dd876d6c8480455"
server_key_iv: "d25e49f6959457564ba3bf346308f730"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.329702 2934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "e26b7ea2be3e489aad171fe1d2eb7cac"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "d0d66aa8566a19e28dd876d6c8480455"
server_key_iv: "d25e49f6959457564ba3bf346308f730"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.331017 2934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:10:35.331717 2949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.331907 2934 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:35.331987 2934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "e26b7ea2be3e489aad171fe1d2eb7cac"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "d0d66aa8566a19e28dd876d6c8480455"
server_key_iv: "d25e49f6959457564ba3bf346308f730"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.332038 2934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:35.349017 2934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:35.351032 2934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:35.351131 2934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:35.351282 2934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:35.351511 2934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:35.351544 2934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.351565 2934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:35.351577 2934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.357123 2934 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:46719
I20260928 23:10:35.357200 3062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:46719 every 8 connection(s)
I20260928 23:10:35.357503 2934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:35.361717 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 2934
I20260928 23:10:35.361835 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:35.362022 16209 external_mini_cluster.cc:1428] Setting key fafc40827c4033c8a7f25cfce2622e7f
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.65@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:35.367048 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.358176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:49511 (local address 127.15.212.126:36817)
0928 23:10:35.358234 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.358236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.358469 (+ 233us) server_negotiation.cc:407] Connection header received
0928 23:10:35.358955 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.358956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.358969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.359023 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:35.359927 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.360427 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.361462 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.361692 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.361769 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.364168 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:35.364170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:35.364170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:35.364177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:35.365611 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.366106 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.366107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.366107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.366157 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.366440 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.366440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.366441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.366536 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:35.366586 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.366917 (+ 331us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.366987 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 23:10:35.367363 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.358152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36817 (local address 127.15.212.65:49511)
0928 23:10:35.358322 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.358385 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.358848 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.359088 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.359092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.359641 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:35.359812 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.359817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.360572 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.360574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.361358 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.361360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.361776 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.361778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.361935 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.362434 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:35.362442 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:35.364072 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:35.365696 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.365702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.365704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.366020 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.366223 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.366224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.366224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.366373 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.366604 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:35.366611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:35.366773 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.366902 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.367073 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260928 23:10:35.367913 3063 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36817
I20260928 23:10:35.368013 3063 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:35.368191 3063 heartbeater.cc:511] Master 127.15.212.126:36817 requested a full tablet report, sending...
I20260928 23:10:35.368669 2872 ts_manager.cc:194] Registered new tserver with Master: e26b7ea2be3e489aad171fe1d2eb7cac (127.15.212.65:46719)
I20260928 23:10:35.369244 2872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.65@KRBTEST.COM'} at 127.15.212.65:49511
WARNING: no policy specified for kudu/127.15.212.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
Entry for principal kudu/127.15.212.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab.
I20260928 23:10:35.388967 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:35.467844 3068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:35.468030 3068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:35.468052 3068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:35.469149 3068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:35.469185 3068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:35.469210 3068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:35.470533 3068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.66.keytab
--principal=kudu/127.15.212.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:35.470697 3068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:35.470872 3068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:35.475704 3073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.475796 3076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.475791 3074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.475998 3068 server_base.cc:1034] running on GCE node
I20260928 23:10:35.476192 3068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:35.476408 3068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:35.477583 3068 hybrid_clock.cc:648] HybridClock initialized: now 1790637035477555 us; error 48 us; skew 500 ppm
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:35.479921 3068 init.cc:377] Logged in from keytab as kudu/127.15.212.66@KRBTEST.COM (short username kudu)
I20260928 23:10:35.480340 3068 webserver.cc:460] Webserver started at http://127.15.212.66:34057/ using document root <none> and password file <none>
I20260928 23:10:35.480587 3068 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:35.480649 3068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:35.480734 3068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:35.481640 3068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "49e9c485ec0a3302d80376caa304c141"
server_key_iv: "6decc90db693472d2feb30699f09dc4f"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.481905 3068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "49e9c485ec0a3302d80376caa304c141"
server_key_iv: "6decc90db693472d2feb30699f09dc4f"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.483179 3068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 23:10:35.483911 3083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.484067 3068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:35.484151 3068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "49e9c485ec0a3302d80376caa304c141"
server_key_iv: "6decc90db693472d2feb30699f09dc4f"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.484223 3068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:35.497023 3068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:35.499146 3068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:35.499243 3068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:35.499388 3068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:35.499681 3068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:35.499714 3068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.499735 3068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:35.499783 3068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.505376 3068 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:35923
I20260928 23:10:35.505468 3196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:35923 every 8 connection(s)
I20260928 23:10:35.505743 3068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.66@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:35.513195 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3068
I20260928 23:10:35.513334 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:35.513541 16209 external_mini_cluster.cc:1428] Setting key 63c3eeafc6201928f2295ce0892eeb6b
I20260928 23:10:35.514321 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.506428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:51315 (local address 127.15.212.126:36817)
0928 23:10:35.506492 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.506494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.506704 (+ 210us) server_negotiation.cc:407] Connection header received
0928 23:10:35.507142 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.507143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.507152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.507206 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:35.508025 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.508572 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.509482 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.509661 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.509709 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.511531 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:35.511532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:35.511533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:35.511539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:35.512905 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.513255 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.513256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.513257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.513339 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.513602 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.513603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.513603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.513740 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:35.513813 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.514145 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.514234 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:10:35.514529 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.506413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36817 (local address 127.15.212.66:51315)
0928 23:10:35.506580 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.506618 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.507055 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.507253 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.507255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.507778 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:35.507948 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.507953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.508646 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.508647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.509408 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.509409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.509728 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.509729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.509829 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.510200 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:35.510208 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:35.511446 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:35.512968 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.512973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.512975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.513170 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.513412 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.513413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.513413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.513524 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.513839 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:35.513846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:35.513993 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.514112 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.514272 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260928 23:10:35.515008 3197 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36817
I20260928 23:10:35.515100 3197 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:35.515277 3197 heartbeater.cc:511] Master 127.15.212.126:36817 requested a full tablet report, sending...
I20260928 23:10:35.515678 2872 ts_manager.cc:194] Registered new tserver with Master: 432e1c5dde0a4f91bdceb7d8a65b9ed9 (127.15.212.66:35923)
I20260928 23:10:35.516191 2872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.66@KRBTEST.COM'} at 127.15.212.66:51315
WARNING: no policy specified for kudu/127.15.212.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
Entry for principal kudu/127.15.212.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab.
I20260928 23:10:35.537081 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:34285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:35.615027 3202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:35.615222 3202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:35.615245 3202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:35.616470 3202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:35.616509 3202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:35.616535 3202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:35.617980 3202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:34285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.67.keytab
--principal=kudu/127.15.212.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:35.618220 3202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:35.618438 3202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:35.623242 3210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:35.623229 3208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.623245 3202 server_base.cc:1034] running on GCE node
W20260928 23:10:35.623229 3207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:35.623715 3202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:35.623935 3202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:35.625067 3202 hybrid_clock.cc:648] HybridClock initialized: now 1790637035625047 us; error 36 us; skew 500 ppm
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:35.627327 3202 init.cc:377] Logged in from keytab as kudu/127.15.212.67@KRBTEST.COM (short username kudu)
I20260928 23:10:35.627777 3202 webserver.cc:460] Webserver started at http://127.15.212.67:38675/ using document root <none> and password file <none>
I20260928 23:10:35.628007 3202 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:35.628083 3202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:35.628235 3202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:35.629384 3202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "6b6e13971e894210a18ec01b559f27eb"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "2ac1e5550807df4b48aaeb3d23a35505"
server_key_iv: "f57fd89162acaf739cb2fc9978eda874"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.629734 3202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "6b6e13971e894210a18ec01b559f27eb"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "2ac1e5550807df4b48aaeb3d23a35505"
server_key_iv: "f57fd89162acaf739cb2fc9978eda874"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.631063 3202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:35.631798 3217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.631971 3202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:35.632073 3202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "6b6e13971e894210a18ec01b559f27eb"
format_stamp: "Formatted at 2026-09-28 23:10:35 on dist-test-slave-x2t8"
server_key: "2ac1e5550807df4b48aaeb3d23a35505"
server_key_iv: "f57fd89162acaf739cb2fc9978eda874"
server_key_version: "encryptionkey@0"
I20260928 23:10:35.632157 3202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:35.652146 3202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:35.654022 3202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:35.654184 3202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:35.654397 3202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:35.654731 3202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:35.654784 3202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.654819 3202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:35.654843 3202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:35.660651 3202 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:40315
I20260928 23:10:35.660704 3330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:40315 every 8 connection(s)
I20260928 23:10:35.660989 3202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:10:35.661278 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3202
I20260928 23:10:35.661387 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:35.661585 16209 external_mini_cluster.cc:1428] Setting key 00ebcf7f222df5616280c11709897f2f
Sep 28 23:10:35 dist-test-slave-x2t8 krb5kdc[2827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.67@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:35.670194 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.661678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:43587 (local address 127.15.212.126:36817)
0928 23:10:35.661745 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.661747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.661983 (+ 236us) server_negotiation.cc:407] Connection header received
0928 23:10:35.662537 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.662538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.662549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.662596 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:35.663518 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.664056 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.665114 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.665333 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.665415 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.667376 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:35.667378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:35.667378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:35.667385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:35.668837 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.669246 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.669247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.669248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.669309 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.669562 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.669563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.669563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.669682 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:35.669754 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.670013 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.670105 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:10:35.670394 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.661665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:36817 (local address 127.15.212.67:43587)
0928 23:10:35.661838 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.661887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.662430 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.662661 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.662665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.663218 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:35.663399 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.663403 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.664157 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.664158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.665010 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.665013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.665425 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.665426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.665546 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.666036 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:10:35.666044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:35.667284 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:35.668935 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.668942 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.668944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.669157 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.669386 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:10:35.669386 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:10:35.669387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:10:35.669495 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:10:35.669759 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:35.669765 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:35.669890 (+ 125us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.669993 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.670156 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260928 23:10:35.670792 3331 heartbeater.cc:347] Connected to a master server at 127.15.212.126:36817
I20260928 23:10:35.670912 3331 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:35.671092 3331 heartbeater.cc:511] Master 127.15.212.126:36817 requested a full tablet report, sending...
I20260928 23:10:35.671432 2872 ts_manager.cc:194] Registered new tserver with Master: 6b6e13971e894210a18ec01b559f27eb (127.15.212.67:40315)
I20260928 23:10:35.671799 2872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.212.67@KRBTEST.COM'} at 127.15.212.67:43587
I20260928 23:10:35.673331 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:35.681859 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.675068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.15.212.126:36817)
0928 23:10:35.675140 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.675142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.675266 (+ 124us) server_negotiation.cc:407] Connection header received
0928 23:10:35.675327 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.675328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.675336 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.675380 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:35.676267 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.676801 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.677956 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.678128 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.678178 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.679810 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:35.679811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:35.679811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:35.679817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:35.681002 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.681301 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.681302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.681302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.681345 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:35.681497 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:35.681497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:35.681498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:35.681610 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:35.681677 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.681759 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.681808 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 23:10:35.683506 2872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:35.685220 2872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:35.691220 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35923 (local address 127.0.0.1:33816)
0928 23:10:35.687225 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.687267 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.687333 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.687761 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.687764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.687766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:35.687837 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.687838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.689170 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.689171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.690077 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.690079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.690947 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.691028 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691041 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.691109 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.691153 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260928 23:10:35.691375 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44794 (local address 127.15.212.67:40315)
0928 23:10:35.687532 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.687536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.687918 (+ 382us) server_negotiation.cc:407] Connection header received
0928 23:10:35.687964 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.687966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.687984 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.688057 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:35.688247 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.688993 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690290 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.690923 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690978 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691021 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.691252 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.691299 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260928 23:10:35.691424 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:40315 (local address 127.0.0.1:44794)
0928 23:10:35.687822 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.687830 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.687873 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.688111 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.688112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.688115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:35.688180 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.688181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.689101 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.689103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.690212 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.690215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.691053 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.691054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.691116 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691121 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.691168 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.691204 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":54,"thread_start_us":37,"threads_started":1}
I20260928 23:10:35.691560 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.15.212.66:35923)
0928 23:10:35.687556 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.687561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.687573 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:10:35.687614 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.687617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.687633 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.687720 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:35.687940 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.689057 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690160 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.690862 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690943 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691010 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.691134 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.691183 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":88,"threads_started":1}
I20260928 23:10:35.691948 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:46719 (local address 127.0.0.1:54272)
0928 23:10:35.687959 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.687967 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.688007 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.688306 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.688307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.688309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:35.688348 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.688350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.689636 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.689637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.690694 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.690696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.691711 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.691712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.691774 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691777 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.691860 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.691895 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":40,"threads_started":1}
I20260928 23:10:35.692027 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.687561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.15.212.65:46719)
0928 23:10:35.687745 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.687748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.688063 (+ 315us) server_negotiation.cc:407] Connection header received
0928 23:10:35.688110 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.688112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.688128 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.688223 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:35.688429 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.689528 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.690776 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.691570 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.691629 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.691681 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.691885 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.691921 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260928 23:10:35.692237 3131 tablet_service.cc:1467] Processing CreateTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 (DEFAULT_TABLE table=test-table [id=3b95ba55280145b0b62a687fc614c410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:35.692239 3265 tablet_service.cc:1467] Processing CreateTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 (DEFAULT_TABLE table=test-table [id=3b95ba55280145b0b62a687fc614c410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:35.692560 3265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26643fc61ba405fb2b7ccb4fd2c5586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:35.692561 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26643fc61ba405fb2b7ccb4fd2c5586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:35.692655 2997 tablet_service.cc:1467] Processing CreateTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 (DEFAULT_TABLE table=test-table [id=3b95ba55280145b0b62a687fc614c410]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:35.692883 2997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26643fc61ba405fb2b7ccb4fd2c5586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:35.695381 3345 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Bootstrap starting.
I20260928 23:10:35.695689 3346 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Bootstrap starting.
I20260928 23:10:35.696065 3345 tablet_bootstrap.cc:654] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:35.696288 3346 tablet_bootstrap.cc:654] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:35.696566 3347 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Bootstrap starting.
I20260928 23:10:35.696759 3345 log.cc:824] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:35.696944 3346 log.cc:824] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:35.697206 3347 tablet_bootstrap.cc:654] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:35.697566 3345 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: No bootstrap required, opened a new log
I20260928 23:10:35.697609 3345 ts_tablet_manager.cc:1397] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:35.697711 3347 log.cc:824] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:35.698336 3347 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: No bootstrap required, opened a new log
I20260928 23:10:35.698375 3347 ts_tablet_manager.cc:1397] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:35.699117 3345 raft_consensus.cc:348] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.699240 3345 raft_consensus.cc:374] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:35.699261 3345 raft_consensus.cc:729] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26b7ea2be3e489aad171fe1d2eb7cac, State: Initialized, Role: FOLLOWER
I20260928 23:10:35.699374 3345 consensus_queue.cc:260] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.699597 3345 ts_tablet_manager.cc:1428] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:35.699709 3063 heartbeater.cc:503] Master 127.15.212.126:36817 was elected leader, sending a full tablet report...
I20260928 23:10:35.699685 3347 raft_consensus.cc:348] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.699838 3347 raft_consensus.cc:374] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:35.699867 3347 raft_consensus.cc:729] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432e1c5dde0a4f91bdceb7d8a65b9ed9, State: Initialized, Role: FOLLOWER
I20260928 23:10:35.700080 3347 consensus_queue.cc:260] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.700390 3346 tablet_bootstrap.cc:492] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: No bootstrap required, opened a new log
I20260928 23:10:35.700390 3197 heartbeater.cc:503] Master 127.15.212.126:36817 was elected leader, sending a full tablet report...
I20260928 23:10:35.700485 3346 ts_tablet_manager.cc:1397] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260928 23:10:35.700383 3347 ts_tablet_manager.cc:1428] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 23:10:35.702093 3346 raft_consensus.cc:348] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.702219 3346 raft_consensus.cc:374] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:35.702251 3346 raft_consensus.cc:729] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6e13971e894210a18ec01b559f27eb, State: Initialized, Role: FOLLOWER
I20260928 23:10:35.702350 3346 consensus_queue.cc:260] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.702610 3346 ts_tablet_manager.cc:1428] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 23:10:35.702633 3331 heartbeater.cc:503] Master 127.15.212.126:36817 was elected leader, sending a full tablet report...
W20260928 23:10:35.756286 3198 tablet.cc:2369] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:35.858227 3064 tablet.cc:2369] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:35.880180 3353 raft_consensus.cc:482] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:35.880298 3353 raft_consensus.cc:504] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.880731 3353 leader_election.cc:288] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 432e1c5dde0a4f91bdceb7d8a65b9ed9 (127.15.212.66:35923), e26b7ea2be3e489aad171fe1d2eb7cac (127.15.212.65:46719)
I20260928 23:10:35.883749 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.880969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:36233 (local address 127.15.212.66:35923)
0928 23:10:35.881026 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.881028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.881057 (+ 29us) server_negotiation.cc:407] Connection header received
0928 23:10:35.881089 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.881090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.881103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.881209 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:35.881396 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.881852 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.882865 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.883410 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.883491 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.883500 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.883648 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.883682 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:10:35.883749 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.880926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:35923 (local address 127.15.212.67:36233)
0928 23:10:35.880968 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.880981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.881025 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.881244 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.881245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.881248 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:35.881316 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.881318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.881934 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.881935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.882766 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.882768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.883489 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.883490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.883568 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.883578 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.883643 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.883684 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":4}
I20260928 23:10:35.883880 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.881053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:46719 (local address 127.15.212.67:53381)
0928 23:10:35.881228 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.881237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:35.881314 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:35.881459 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:35.881460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:35.881461 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:35.881540 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.881541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.882179 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.882180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.882944 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:35.882946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.883606 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.883606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:35.883679 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.883683 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 23:10:35.883755 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:10:35.883791 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2,"thread_start_us":117,"threads_started":1}
I20260928 23:10:35.883868 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.881135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:53381 (local address 127.15.212.65:46719)
0928 23:10:35.881206 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.881208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.881304 (+ 96us) server_negotiation.cc:407] Connection header received
0928 23:10:35.881367 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.881368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.881379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.881445 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:35.881612 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.882103 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.883021 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.883525 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.883605 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.883613 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.883764 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.883797 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 23:10:35.884244 3017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26643fc61ba405fb2b7ccb4fd2c5586" candidate_uuid: "6b6e13971e894210a18ec01b559f27eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" is_pre_election: true
I20260928 23:10:35.884311 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26643fc61ba405fb2b7ccb4fd2c5586" candidate_uuid: "6b6e13971e894210a18ec01b559f27eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" is_pre_election: true
I20260928 23:10:35.884410 3017 raft_consensus.cc:2454] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b6e13971e894210a18ec01b559f27eb in term 0.
I20260928 23:10:35.884462 3151 raft_consensus.cc:2454] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b6e13971e894210a18ec01b559f27eb in term 0.
I20260928 23:10:35.884658 3221 leader_election.cc:302] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432e1c5dde0a4f91bdceb7d8a65b9ed9, 6b6e13971e894210a18ec01b559f27eb; no voters:
I20260928 23:10:35.884801 3353 raft_consensus.cc:2781] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:35.884886 3353 raft_consensus.cc:482] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:35.884917 3353 raft_consensus.cc:3037] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:35.885628 3353 raft_consensus.cc:504] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.885761 3353 leader_election.cc:288] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [CANDIDATE]: Term 1 election: Requested vote from peers 432e1c5dde0a4f91bdceb7d8a65b9ed9 (127.15.212.66:35923), e26b7ea2be3e489aad171fe1d2eb7cac (127.15.212.65:46719)
I20260928 23:10:35.885936 3151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26643fc61ba405fb2b7ccb4fd2c5586" candidate_uuid: "6b6e13971e894210a18ec01b559f27eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9"
I20260928 23:10:35.885982 3017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26643fc61ba405fb2b7ccb4fd2c5586" candidate_uuid: "6b6e13971e894210a18ec01b559f27eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac"
I20260928 23:10:35.886056 3017 raft_consensus.cc:3037] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:35.886042 3151 raft_consensus.cc:3037] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:35.886951 3151 raft_consensus.cc:2454] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b6e13971e894210a18ec01b559f27eb in term 1.
I20260928 23:10:35.886951 3017 raft_consensus.cc:2454] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b6e13971e894210a18ec01b559f27eb in term 1.
I20260928 23:10:35.887122 3221 leader_election.cc:302] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b6e13971e894210a18ec01b559f27eb, e26b7ea2be3e489aad171fe1d2eb7cac; no voters:
I20260928 23:10:35.887204 3353 raft_consensus.cc:2781] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:35.887341 3353 raft_consensus.cc:686] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 1 LEADER]: Becoming Leader. State: Replica: 6b6e13971e894210a18ec01b559f27eb, State: Running, Role: LEADER
I20260928 23:10:35.887436 3353 consensus_queue.cc:237] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } }
I20260928 23:10:35.888324 2872 catalog_manager.cc:5495] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b6e13971e894210a18ec01b559f27eb (127.15.212.67). New cstate: current_term: 1 leader_uuid: "6b6e13971e894210a18ec01b559f27eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b6e13971e894210a18ec01b559f27eb" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 } health_report { overall_health: UNKNOWN } } }
W20260928 23:10:35.911547 3332 tablet.cc:2369] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:35.917218 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.914794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.15.212.67:40315)
0928 23:10:35.914856 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:35.914858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:35.914866 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:10:35.914898 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:35.914899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:35.914931 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:35.915006 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:35.915165 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.915640 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.916394 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:35.916601 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:35.916675 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:35.916856 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:35.916955 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:35.917047 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:35.917097 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 23:10:35.917119 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:10:35.918901 3151 raft_consensus.cc:1261] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 1 FOLLOWER]: Refusing update from remote peer 6b6e13971e894210a18ec01b559f27eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:35.918921 3017 raft_consensus.cc:1261] T e26643fc61ba405fb2b7ccb4fd2c5586 P e26b7ea2be3e489aad171fe1d2eb7cac [term 1 FOLLOWER]: Refusing update from remote peer 6b6e13971e894210a18ec01b559f27eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:35.919319 3355 consensus_queue.cc:1035] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e26b7ea2be3e489aad171fe1d2eb7cac" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:35.919436 3353 consensus_queue.cc:1035] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "432e1c5dde0a4f91bdceb7d8a65b9ed9" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 35923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:35.921996 3358 mvcc.cc:205] Tried to move back new op lower bound from 7334449299121381376 to 7334449298995724288. Current Snapshot: MvccSnapshot[applied={T|T < 7334449299121381376}]
I20260928 23:10:35.922096 3359 mvcc.cc:205] Tried to move back new op lower bound from 7334449299121381376 to 7334449298995724288. Current Snapshot: MvccSnapshot[applied={T|T < 7334449299121381376}]
I20260928 23:10:35.922464 3360 mvcc.cc:205] Tried to move back new op lower bound from 7334449299121381376 to 7334449298995724288. Current Snapshot: MvccSnapshot[applied={T|T < 7334449299121381376}]
I20260928 23:10:35.924042 2871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:35.924135 2871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:35.924928 2871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c55b9537ddbc43a8a550797746ccb4ed: Sending DeleteTablet for 3 replicas of tablet e26643fc61ba405fb2b7ccb4fd2c5586
I20260928 23:10:35.925238 3131 tablet_service.cc:1514] Processing DeleteTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:33816
I20260928 23:10:35.925475 3265 tablet_service.cc:1514] Processing DeleteTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:44794
I20260928 23:10:35.925508 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2934
I20260928 23:10:35.925675 2997 tablet_service.cc:1514] Processing DeleteTablet for tablet e26643fc61ba405fb2b7ccb4fd2c5586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:35 UTC) from {username='kudu', principal='kudu/127.15.212.126@KRBTEST.COM'} at 127.0.0.1:54272
I20260928 23:10:35.926050 3367 tablet_replica.cc:330] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: stopping tablet replica
I20260928 23:10:35.926147 3367 raft_consensus.cc:2229] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:35.926247 3367 raft_consensus.cc:2258] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:35.927058 3367 ts_tablet_manager.cc:1905] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:35.929071 3367 ts_tablet_manager.cc:1918] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:35.929145 3367 log.cc:1196] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/e26643fc61ba405fb2b7ccb4fd2c5586
I20260928 23:10:35.929402 3367 ts_tablet_manager.cc:1939] T e26643fc61ba405fb2b7ccb4fd2c5586 P 432e1c5dde0a4f91bdceb7d8a65b9ed9: Deleting consensus metadata
I20260928 23:10:35.929787 2859 catalog_manager.cc:4841] TS 432e1c5dde0a4f91bdceb7d8a65b9ed9 (127.15.212.66:35923): tablet e26643fc61ba405fb2b7ccb4fd2c5586 (table test-table [id=3b95ba55280145b0b62a687fc614c410]) successfully deleted
W20260928 23:10:35.931114 2859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:46719 (error 108)
I20260928 23:10:35.931365 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3068
I20260928 23:10:35.932000 3368 tablet_replica.cc:330] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: stopping tablet replica
I20260928 23:10:35.932098 3368 raft_consensus.cc:2229] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:10:35.932207 3368 raft_consensus.cc:2258] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:35.932592 3368 ts_tablet_manager.cc:1905] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:35.933082 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:35.932647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:46719 (local address 127.0.0.1:54288)
0928 23:10:35.933005 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:35.933049 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336}
W20260928 23:10:35.933219 2859 catalog_manager.cc:4584] TS e26b7ea2be3e489aad171fe1d2eb7cac (127.15.212.65:46719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e26643fc61ba405fb2b7ccb4fd2c5586: Network error: Client connection negotiation failed: client connection to 127.15.212.65:46719: connect: Connection refused (error 111)
I20260928 23:10:35.934031 3368 ts_tablet_manager.cc:1918] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:35.934088 3368 log.cc:1196] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/e26643fc61ba405fb2b7ccb4fd2c5586
I20260928 23:10:35.934401 3368 ts_tablet_manager.cc:1939] T e26643fc61ba405fb2b7ccb4fd2c5586 P 6b6e13971e894210a18ec01b559f27eb: Deleting consensus metadata
I20260928 23:10:35.934731 2859 catalog_manager.cc:4841] TS 6b6e13971e894210a18ec01b559f27eb (127.15.212.67:40315): tablet e26643fc61ba405fb2b7ccb4fd2c5586 (table test-table [id=3b95ba55280145b0b62a687fc614c410]) successfully deleted
I20260928 23:10:35.938495 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3202
I20260928 23:10:35.943702 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 2840
2026-09-28T23:10:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2029 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T23:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:35Z Disabled control of system clock
I20260928 23:10:35.970250 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32967
--webserver_interface=127.15.212.126
--webserver_port=0
--builtin_ntp_servers=127.15.212.84:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:32967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:36.049701 3373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:36.049903 3373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:36.049923 3373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:36.051054 3373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:36.051095 3373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:36.051107 3373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:36.051120 3373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:36.051167 3373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:36.052528 3373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:32967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:32967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:36.052757 3373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:36.053035 3373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:36.057570 3378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.057585 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.057570 3381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:36.057861 3373 server_base.cc:1034] running on GCE node
I20260928 23:10:36.058080 3373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:36.058303 3373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:36.059454 3373 hybrid_clock.cc:648] HybridClock initialized: now 1790637036059436 us; error 33 us; skew 500 ppm
I20260928 23:10:36.060665 3373 webserver.cc:460] Webserver started at http://127.15.212.126:40103/ using document root <none> and password file <none>
I20260928 23:10:36.060897 3373 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:36.060959 3373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:36.061112 3373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:36.062099 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "12169f5777b24cb88767eaba616a9921"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20ef5041bf7213e6e47041fba582e62c"
server_key_iv: "ddc7cbb27742146fad3c7c5b861c5593"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.062439 3373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "12169f5777b24cb88767eaba616a9921"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20ef5041bf7213e6e47041fba582e62c"
server_key_iv: "ddc7cbb27742146fad3c7c5b861c5593"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.063731 3373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:36.064409 3387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.064586 3373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:36.064682 3373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "12169f5777b24cb88767eaba616a9921"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20ef5041bf7213e6e47041fba582e62c"
server_key_iv: "ddc7cbb27742146fad3c7c5b861c5593"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.064760 3373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:36.094017 3373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:36.094350 3373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:36.094516 3373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:36.098515 3373 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:32967
I20260928 23:10:36.098662 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:32967 every 8 connection(s)
I20260928 23:10:36.098940 3373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:36.099865 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:36.102118 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: Bootstrap starting.
I20260928 23:10:36.102685 3440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:36.102978 3440 log.cc:824] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:36.103626 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: No bootstrap required, opened a new log
I20260928 23:10:36.104517 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3373
I20260928 23:10:36.104629 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:36.104843 16209 external_mini_cluster.cc:1428] Setting key 0ac57a6b955839ccce5a6bd18fa8cc06
I20260928 23:10:36.105886 3440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } }
I20260928 23:10:36.106051 3440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:36.106091 3440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12169f5777b24cb88767eaba616a9921, State: Initialized, Role: FOLLOWER
I20260928 23:10:36.106222 3440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } }
I20260928 23:10:36.106315 3440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:36.106340 3440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:36.106407 3440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:36.106963 3440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } }
I20260928 23:10:36.107053 3440 leader_election.cc:302] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12169f5777b24cb88767eaba616a9921; no voters:
I20260928 23:10:36.107229 3440 leader_election.cc:288] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:36.107267 3445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:36.107401 3445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [term 1 LEADER]: Becoming Leader. State: Replica: 12169f5777b24cb88767eaba616a9921, State: Running, Role: LEADER
I20260928 23:10:36.107488 3440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:36.107483 3445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } }
I20260928 23:10:36.110108 3446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12169f5777b24cb88767eaba616a9921. Latest consensus state: current_term: 1 leader_uuid: "12169f5777b24cb88767eaba616a9921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } } }
I20260928 23:10:36.110209 3446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:36.109268 3447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12169f5777b24cb88767eaba616a9921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12169f5777b24cb88767eaba616a9921" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 32967 } } }
W20260928 23:10:36.109418 3460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:10:36.110431 3460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:10:36.110507 3447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:36.110560 3462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:36.110700 3462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:36.111351 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.105357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.15.212.126:32967)
0928 23:10:36.105589 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.105597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.105621 (+ 24us) server_negotiation.cc:407] Connection header received
0928 23:10:36.105868 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.105874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.105934 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.106041 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:36.106282 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.107908 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.109130 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.109632 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.109780 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.110522 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:36.110525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:36.110526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:36.110723 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:36.110764 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:36.110778 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:36.110849 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.111048 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.111130 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260928 23:10:36.112177 3462 catalog_manager.cc:1347] Generated new cluster ID: a374a80a1748499fae71f0acc469f539
I20260928 23:10:36.112241 3462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:36.118477 3462 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:36.118960 3462 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:36.127887 3462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: Generated new TSK 0
I20260928 23:10:36.128042 3462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:36.131942 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.65:0
--local_ip_for_outbound_sockets=127.15.212.65
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--builtin_ntp_servers=127.15.212.84:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:36.215099 3464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:36.215322 3464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:36.215356 3464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:36.216530 3464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:36.216583 3464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:36.216624 3464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.65
I20260928 23:10:36.217988 3464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.212.65
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.65
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:36.218240 3464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:36.218562 3464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:36.224201 3470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.224203 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.224172 3469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:36.224716 3464 server_base.cc:1034] running on GCE node
I20260928 23:10:36.224947 3464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:36.225214 3464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:36.226372 3464 hybrid_clock.cc:648] HybridClock initialized: now 1790637036226355 us; error 33 us; skew 500 ppm
I20260928 23:10:36.227586 3464 webserver.cc:460] Webserver started at http://127.15.212.65:34929/ using document root <none> and password file <none>
I20260928 23:10:36.227834 3464 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:36.227921 3464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:36.228080 3464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:36.229229 3464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data/instance:
uuid: "a85b2f780ca44d2e98a5398f8157eaf1"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20b852016aecf2a80e28f908ea2f4f83"
server_key_iv: "ed44ac026b0765791ee4aef99e851304"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.229583 3464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance:
uuid: "a85b2f780ca44d2e98a5398f8157eaf1"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20b852016aecf2a80e28f908ea2f4f83"
server_key_iv: "ed44ac026b0765791ee4aef99e851304"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.230997 3464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:36.231779 3478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.231951 3464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:36.232053 3464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal
uuid: "a85b2f780ca44d2e98a5398f8157eaf1"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "20b852016aecf2a80e28f908ea2f4f83"
server_key_iv: "ed44ac026b0765791ee4aef99e851304"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.232121 3464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:36.242448 3464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:36.242671 3464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:36.242766 3464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:36.242920 3464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:36.243172 3464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:36.243202 3464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.243222 3464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:36.243235 3464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.248960 3464 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.65:33429
I20260928 23:10:36.249017 3591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.65:33429 every 8 connection(s)
I20260928 23:10:36.249303 3464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/data/info.pb
I20260928 23:10:36.254374 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.249959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.65:37545 (local address 127.15.212.126:32967)
0928 23:10:36.250002 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.250004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.250207 (+ 203us) server_negotiation.cc:407] Connection header received
0928 23:10:36.250727 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.250728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.250743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.250791 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:36.251409 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.251895 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.252968 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.253178 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.253261 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.254095 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:36.254097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:36.254097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:36.254102 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:36.254115 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:36.254119 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:36.254168 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.254270 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.254297 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:36.254581 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.249930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32967 (local address 127.15.212.65:37545)
0928 23:10:36.250098 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.250128 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.250619 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.250852 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.250856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.250865 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:36.251319 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.251323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.251995 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.251997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.252878 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.252881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.253271 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.253272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.253420 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.253944 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:36.253944 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:36.253970 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:36.253997 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:36.254181 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:36.254185 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:36.254188 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.254271 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.254367 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260928 23:10:36.255043 3592 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32967
I20260928 23:10:36.255152 3592 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:36.255322 3592 heartbeater.cc:511] Master 127.15.212.126:32967 requested a full tablet report, sending...
I20260928 23:10:36.255786 3404 ts_manager.cc:194] Registered new tserver with Master: a85b2f780ca44d2e98a5398f8157eaf1 (127.15.212.65:33429)
I20260928 23:10:36.256325 3404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.65:37545
I20260928 23:10:36.259142 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3464
I20260928 23:10:36.259217 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-0/wal/instance
I20260928 23:10:36.259366 16209 external_mini_cluster.cc:1428] Setting key 0a92782b40c6d8822402d322c00565a9
I20260928 23:10:36.260171 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.66:0
--local_ip_for_outbound_sockets=127.15.212.66
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--builtin_ntp_servers=127.15.212.84:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 23:10:36.338269 3595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:36.338456 3595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:36.338477 3595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:36.339797 3595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:36.339836 3595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:36.339861 3595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.66
I20260928 23:10:36.341169 3595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.212.66
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.66
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:36.341332 3595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:36.341543 3595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:10:36.346346 3595 server_base.cc:1034] running on GCE node
W20260928 23:10:36.346304 3601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.346292 3603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.346271 3600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:36.346776 3595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:36.347023 3595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:36.348183 3595 hybrid_clock.cc:648] HybridClock initialized: now 1790637036348175 us; error 39 us; skew 500 ppm
I20260928 23:10:36.349478 3595 webserver.cc:460] Webserver started at http://127.15.212.66:34945/ using document root <none> and password file <none>
I20260928 23:10:36.349720 3595 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:36.349788 3595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:36.349920 3595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:36.350894 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data/instance:
uuid: "afc868b172e148cbaeeaed45813c6a17"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "e01cd61e958c2952535eb1d5fdc8ec8b"
server_key_iv: "baa1d84924c3f8c4e39e02ef1f606860"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.351249 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance:
uuid: "afc868b172e148cbaeeaed45813c6a17"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "e01cd61e958c2952535eb1d5fdc8ec8b"
server_key_iv: "baa1d84924c3f8c4e39e02ef1f606860"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.352504 3595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:36.353194 3609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.353340 3595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:36.353448 3595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal
uuid: "afc868b172e148cbaeeaed45813c6a17"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "e01cd61e958c2952535eb1d5fdc8ec8b"
server_key_iv: "baa1d84924c3f8c4e39e02ef1f606860"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.353534 3595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:36.375984 3595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:36.376272 3595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:36.376426 3595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:36.376698 3595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:36.377012 3595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:36.377065 3595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.377105 3595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:36.377134 3595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.382700 3595 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.66:44571
I20260928 23:10:36.382756 3722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.66:44571 every 8 connection(s)
I20260928 23:10:36.383045 3595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/data/info.pb
I20260928 23:10:36.384634 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3595
I20260928 23:10:36.384740 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal/instance
I20260928 23:10:36.384976 16209 external_mini_cluster.cc:1428] Setting key ca36fc34bfa6037879749bffd7e2c6a1
I20260928 23:10:36.385932 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.212.67:0
--local_ip_for_outbound_sockets=127.15.212.67
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--builtin_ntp_servers=127.15.212.84:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 23:10:36.388422 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.383700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.66:59255 (local address 127.15.212.126:32967)
0928 23:10:36.383746 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.383750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.383968 (+ 218us) server_negotiation.cc:407] Connection header received
0928 23:10:36.384416 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.384417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.384426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.384498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:36.385364 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.385916 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.386935 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.387129 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.387205 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.388060 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:36.388062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:36.388063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:36.388068 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:36.388082 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:36.388087 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:36.388138 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.388276 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.388303 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:10:36.388540 3725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.383677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32967 (local address 127.15.212.66:59255)
0928 23:10:36.383849 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.383887 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.384326 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.384589 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.384594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.384606 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:36.385261 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.385267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.386021 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.386023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.386832 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.386835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.387214 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.387215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.387355 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.387899 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:36.387900 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:36.387927 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:36.387949 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:36.388176 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:36.388183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:36.388186 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.388256 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.388335 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260928 23:10:36.389017 3723 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32967
I20260928 23:10:36.389093 3723 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:36.389273 3723 heartbeater.cc:511] Master 127.15.212.126:32967 requested a full tablet report, sending...
I20260928 23:10:36.389683 3404 ts_manager.cc:194] Registered new tserver with Master: afc868b172e148cbaeeaed45813c6a17 (127.15.212.66:44571)
I20260928 23:10:36.390208 3404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.66:59255
W20260928 23:10:36.465886 3726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:36.466073 3726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:36.466094 3726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:36.467101 3726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:36.467132 3726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:36.467181 3726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.212.67
I20260928 23:10:36.468614 3726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.212.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.212.67
--webserver_port=0
--tserver_master_addrs=127.15.212.126:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.212.67
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:36.468786 3726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:36.469065 3726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:36.473816 3732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.473829 3734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:36.473846 3731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:36.474061 3726 server_base.cc:1034] running on GCE node
I20260928 23:10:36.474268 3726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:36.474510 3726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:36.475683 3726 hybrid_clock.cc:648] HybridClock initialized: now 1790637036475668 us; error 25 us; skew 500 ppm
I20260928 23:10:36.476943 3726 webserver.cc:460] Webserver started at http://127.15.212.67:35917/ using document root <none> and password file <none>
I20260928 23:10:36.477203 3726 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:36.477276 3726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:36.477432 3726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:36.478471 3726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data/instance:
uuid: "bd6973eee85541c38945c06dbfc9daf4"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "a6ccf49dfc39c2a03fa35387b26a96d1"
server_key_iv: "5b53b81478eab55aa7b0adee7f235b96"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.478813 3726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance:
uuid: "bd6973eee85541c38945c06dbfc9daf4"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "a6ccf49dfc39c2a03fa35387b26a96d1"
server_key_iv: "5b53b81478eab55aa7b0adee7f235b96"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.480011 3726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:36.480741 3740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.480935 3726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:36.481051 3726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal
uuid: "bd6973eee85541c38945c06dbfc9daf4"
format_stamp: "Formatted at 2026-09-28 23:10:36 on dist-test-slave-x2t8"
server_key: "a6ccf49dfc39c2a03fa35387b26a96d1"
server_key_iv: "5b53b81478eab55aa7b0adee7f235b96"
server_key_version: "encryptionkey@0"
I20260928 23:10:36.481133 3726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:36.487587 3726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:36.487844 3726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:36.487989 3726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:36.488197 3726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 23:10:36.488564 3726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 23:10:36.488615 3726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.488653 3726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 23:10:36.488680 3726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:36.494017 3726 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.67:39631
I20260928 23:10:36.494078 3853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.67:39631 every 8 connection(s)
I20260928 23:10:36.494388 3726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/data/info.pb
I20260928 23:10:36.499076 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.495058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:49605 (local address 127.15.212.126:32967)
0928 23:10:36.495098 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.495100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.495314 (+ 214us) server_negotiation.cc:407] Connection header received
0928 23:10:36.495732 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.495733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.495741 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.495796 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:36.496363 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.496895 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.497916 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.498100 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.498149 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.498895 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:36.498896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:36.498896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:36.498900 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:36.498909 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:36.498912 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:36.498948 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.499001 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.499023 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:10:36.499298 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.495019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:32967 (local address 127.15.212.67:49605)
0928 23:10:36.495193 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.495230 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.495657 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.495841 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.495844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.495852 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:10:36.496302 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.496306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.496985 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.496986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.497840 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.497842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.498167 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.498167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.498269 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.498782 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 23:10:36.498782 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:10:36.498807 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 23:10:36.498828 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:10:36.498950 (+ 122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:10:36.498952 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:10:36.498954 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.499003 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.499069 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260928 23:10:36.499737 3854 heartbeater.cc:347] Connected to a master server at 127.15.212.126:32967
I20260928 23:10:36.499833 3854 heartbeater.cc:464] Registering TS with master...
I20260928 23:10:36.500001 3854 heartbeater.cc:511] Master 127.15.212.126:32967 requested a full tablet report, sending...
I20260928 23:10:36.500356 3404 ts_manager.cc:194] Registered new tserver with Master: bd6973eee85541c38945c06dbfc9daf4 (127.15.212.67:39631)
I20260928 23:10:36.500728 3404 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.212.67:49605
I20260928 23:10:36.501259 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3726
I20260928 23:10:36.501335 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal/instance
I20260928 23:10:36.501543 16209 external_mini_cluster.cc:1428] Setting key 8ce6deb7d613e88a158979ad9840bcfb
I20260928 23:10:36.502287 16209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 23:10:36.506398 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.503558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.15.212.126:32967)
0928 23:10:36.503631 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.503633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.503727 (+ 94us) server_negotiation.cc:407] Connection header received
0928 23:10:36.503787 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.503788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.503800 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.503864 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:36.504033 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.504515 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.505252 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.505436 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.505509 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.506154 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:36.506155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:36.506156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:36.506160 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:36.506171 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:36.506175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:36.506242 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.506301 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.506322 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 23:10:36.507936 3404 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 23:10:36.509613 3404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 23:10:36.515376 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:44571 (local address 127.0.0.1:37708)
0928 23:10:36.511956 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.511965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.512007 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.512189 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.512190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.512193 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:36.512241 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.512243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513199 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.513200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.514297 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.514300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515122 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.515190 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515201 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.515262 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.515311 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":95,"threads_started":1}
I20260928 23:10:36.515395 3867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.15.212.66:44571)
0928 23:10:36.512033 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.512038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.512048 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:10:36.512084 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.512088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.512113 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.512190 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:36.512328 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513105 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.514390 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515024 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515121 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515202 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.515285 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.515322 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
I20260928 23:10:36.515632 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.67:39631 (local address 127.0.0.1:55714)
0928 23:10:36.511661 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.511699 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.511758 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.512129 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.512133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.512135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:36.512212 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.512214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513639 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.513639 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.514526 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.514528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515392 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.515452 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515467 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.515499 (+ 32us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.515562 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":12,"thread_start_us":53,"threads_started":1}
I20260928 23:10:36.515805 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:33429 (local address 127.0.0.1:52552)
0928 23:10:36.511864 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.511874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.511909 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.512283 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.512283 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.512285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:36.512318 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.512319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513622 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.513623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.514708 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.514710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515587 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.515643 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515658 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.515687 (+ 29us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.515724 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":28,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
I20260928 23:10:36.516023 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.15.212.67:39631)
0928 23:10:36.511942 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.511946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.511954 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:10:36.511983 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.511985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.511997 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.512107 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:36.512395 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513527 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.514619 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515291 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515804 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515894 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.515919 (+ 25us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.515951 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":78,"threads_started":1}
I20260928 23:10:36.516148 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.511751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.15.212.65:33429)
0928 23:10:36.511902 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.511907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.511919 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:10:36.511968 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.511970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.511985 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.512107 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:36.512375 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.513527 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.514790 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.515485 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.515898 (+ 413us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.515993 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.516017 (+ 24us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.516058 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260928 23:10:36.516234 3657 tablet_service.cc:1467] Processing CreateTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 (DEFAULT_TABLE table=test-table [id=ddaafc8e73f140959fc4baf9c1fcd856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:36.516626 3657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:36.516621 3788 tablet_service.cc:1467] Processing CreateTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 (DEFAULT_TABLE table=test-table [id=ddaafc8e73f140959fc4baf9c1fcd856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:36.516912 3788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:36.519124 3868 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Bootstrap starting.
I20260928 23:10:36.519150 3869 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Bootstrap starting.
I20260928 23:10:36.519814 3869 tablet_bootstrap.cc:654] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:36.519852 3868 tablet_bootstrap.cc:654] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:36.519903 3526 tablet_service.cc:1467] Processing CreateTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 (DEFAULT_TABLE table=test-table [id=ddaafc8e73f140959fc4baf9c1fcd856]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 23:10:36.520150 3869 log.cc:824] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:36.520187 3868 log.cc:824] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:36.520252 3526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:36.520980 3868 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: No bootstrap required, opened a new log
I20260928 23:10:36.521060 3868 ts_tablet_manager.cc:1397] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:36.523582 3872 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Bootstrap starting.
I20260928 23:10:36.523682 3868 raft_consensus.cc:348] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.523845 3868 raft_consensus.cc:374] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:36.523890 3868 raft_consensus.cc:729] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afc868b172e148cbaeeaed45813c6a17, State: Initialized, Role: FOLLOWER
I20260928 23:10:36.524024 3868 consensus_queue.cc:260] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.524155 3869 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: No bootstrap required, opened a new log
I20260928 23:10:36.524230 3872 tablet_bootstrap.cc:654] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:36.524231 3869 ts_tablet_manager.cc:1397] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 23:10:36.524273 3868 ts_tablet_manager.cc:1428] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 23:10:36.524387 3723 heartbeater.cc:503] Master 127.15.212.126:32967 was elected leader, sending a full tablet report...
I20260928 23:10:36.524578 3872 log.cc:824] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:36.525331 3872 tablet_bootstrap.cc:492] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: No bootstrap required, opened a new log
I20260928 23:10:36.525409 3872 ts_tablet_manager.cc:1397] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 23:10:36.526921 3869 raft_consensus.cc:348] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.527084 3869 raft_consensus.cc:374] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:36.527118 3869 raft_consensus.cc:729] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6973eee85541c38945c06dbfc9daf4, State: Initialized, Role: FOLLOWER
I20260928 23:10:36.527230 3869 consensus_queue.cc:260] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.527460 3869 ts_tablet_manager.cc:1428] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 23:10:36.527504 3854 heartbeater.cc:503] Master 127.15.212.126:32967 was elected leader, sending a full tablet report...
I20260928 23:10:36.527835 3872 raft_consensus.cc:348] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.527966 3872 raft_consensus.cc:374] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:36.528010 3872 raft_consensus.cc:729] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85b2f780ca44d2e98a5398f8157eaf1, State: Initialized, Role: FOLLOWER
I20260928 23:10:36.528113 3872 consensus_queue.cc:260] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.528402 3872 ts_tablet_manager.cc:1428] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 23:10:36.528561 3592 heartbeater.cc:503] Master 127.15.212.126:32967 was elected leader, sending a full tablet report...
W20260928 23:10:36.633574 3724 tablet.cc:2369] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:36.744930 3855 tablet.cc:2369] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 23:10:36.749884 3593 tablet.cc:2369] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 23:10:36.785346 3875 raft_consensus.cc:482] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 23:10:36.785495 3875 raft_consensus.cc:504] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.785861 3875 leader_election.cc:288] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a85b2f780ca44d2e98a5398f8157eaf1 (127.15.212.65:33429), afc868b172e148cbaeeaed45813c6a17 (127.15.212.66:44571)
I20260928 23:10:36.789604 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.786049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:33429 (local address 127.15.212.67:48679)
0928 23:10:36.786091 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.786103 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.786147 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.786342 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.786344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.786346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:36.786423 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.786426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.787360 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.787360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.788585 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.788588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.789333 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.789334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.789416 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.789421 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.789481 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.789532 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":11}
I20260928 23:10:36.789621 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.786148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:48679 (local address 127.15.212.65:33429)
0928 23:10:36.786190 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.786192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.786201 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:10:36.786228 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.786229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.786239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.786297 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:36.786516 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.787277 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.788687 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.789233 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.789313 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.789322 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.789511 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.789553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:10:36.789621 3867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.786134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.212.67:51151 (local address 127.15.212.66:44571)
0928 23:10:36.786202 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.786204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.786492 (+ 288us) server_negotiation.cc:407] Connection header received
0928 23:10:36.786525 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.786527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.786537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.786594 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 23:10:36.786727 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.787406 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.788685 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.789233 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.789314 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.789322 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.789510 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.789550 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 23:10:36.789610 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.786049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.66:44571 (local address 127.15.212.67:51151)
0928 23:10:36.786376 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.786387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:10:36.786464 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:10:36.786605 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:10:36.786605 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:10:36.786606 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 23:10:36.786656 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.786658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.787485 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.787485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.788591 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:10:36.788593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.789334 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.789334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:10:36.789416 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.789422 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 23:10:36.789481 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 23:10:36.789522 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":150,"thread_start_us":102,"threads_started":1}
I20260928 23:10:36.789973 3546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4261dd41cf7a4ba9a1671c5d6b64fe82" candidate_uuid: "bd6973eee85541c38945c06dbfc9daf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" is_pre_election: true
I20260928 23:10:36.789987 3677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4261dd41cf7a4ba9a1671c5d6b64fe82" candidate_uuid: "bd6973eee85541c38945c06dbfc9daf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afc868b172e148cbaeeaed45813c6a17" is_pre_election: true
I20260928 23:10:36.790149 3546 raft_consensus.cc:2454] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6973eee85541c38945c06dbfc9daf4 in term 0.
I20260928 23:10:36.790162 3677 raft_consensus.cc:2454] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6973eee85541c38945c06dbfc9daf4 in term 0.
I20260928 23:10:36.790423 3743 leader_election.cc:302] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afc868b172e148cbaeeaed45813c6a17, bd6973eee85541c38945c06dbfc9daf4; no voters:
I20260928 23:10:36.790555 3875 raft_consensus.cc:2781] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 23:10:36.790611 3875 raft_consensus.cc:482] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 23:10:36.790632 3875 raft_consensus.cc:3037] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:36.791306 3875 raft_consensus.cc:504] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.791448 3875 leader_election.cc:288] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [CANDIDATE]: Term 1 election: Requested vote from peers a85b2f780ca44d2e98a5398f8157eaf1 (127.15.212.65:33429), afc868b172e148cbaeeaed45813c6a17 (127.15.212.66:44571)
I20260928 23:10:36.791597 3546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4261dd41cf7a4ba9a1671c5d6b64fe82" candidate_uuid: "bd6973eee85541c38945c06dbfc9daf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85b2f780ca44d2e98a5398f8157eaf1"
I20260928 23:10:36.791612 3677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4261dd41cf7a4ba9a1671c5d6b64fe82" candidate_uuid: "bd6973eee85541c38945c06dbfc9daf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afc868b172e148cbaeeaed45813c6a17"
I20260928 23:10:36.791683 3546 raft_consensus.cc:3037] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:36.791683 3677 raft_consensus.cc:3037] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:36.792600 3546 raft_consensus.cc:2454] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6973eee85541c38945c06dbfc9daf4 in term 1.
I20260928 23:10:36.792600 3677 raft_consensus.cc:2454] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6973eee85541c38945c06dbfc9daf4 in term 1.
I20260928 23:10:36.792769 3743 leader_election.cc:302] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afc868b172e148cbaeeaed45813c6a17, bd6973eee85541c38945c06dbfc9daf4; no voters:
I20260928 23:10:36.792865 3875 raft_consensus.cc:2781] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:36.793030 3875 raft_consensus.cc:686] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 1 LEADER]: Becoming Leader. State: Replica: bd6973eee85541c38945c06dbfc9daf4, State: Running, Role: LEADER
I20260928 23:10:36.793124 3875 consensus_queue.cc:237] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } }
I20260928 23:10:36.793912 3404 catalog_manager.cc:5495] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd6973eee85541c38945c06dbfc9daf4 (127.15.212.67). New cstate: current_term: 1 leader_uuid: "bd6973eee85541c38945c06dbfc9daf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd6973eee85541c38945c06dbfc9daf4" member_type: VOTER last_known_addr { host: "127.15.212.67" port: 39631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 } health_report { overall_health: UNKNOWN } } }
I20260928 23:10:36.868722 3865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.866148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.15.212.67:39631)
0928 23:10:36.866191 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:36.866193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:36.866201 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:10:36.866272 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:36.866273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:36.866288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:36.866410 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:10:36.866586 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.867092 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.867809 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:36.868000 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:36.868054 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:36.868235 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:10:36.868328 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:10:36.868389 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:36.868620 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 23:10:36.868641 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:10:36.870527 3546 raft_consensus.cc:1261] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 1 FOLLOWER]: Refusing update from remote peer bd6973eee85541c38945c06dbfc9daf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:36.870514 3677 raft_consensus.cc:1261] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 1 FOLLOWER]: Refusing update from remote peer bd6973eee85541c38945c06dbfc9daf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:10:36.870886 3878 consensus_queue.cc:1035] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afc868b172e148cbaeeaed45813c6a17" member_type: VOTER last_known_addr { host: "127.15.212.66" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:36.871068 3875 consensus_queue.cc:1035] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85b2f780ca44d2e98a5398f8157eaf1" member_type: VOTER last_known_addr { host: "127.15.212.65" port: 33429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:10:36.873679 3883 mvcc.cc:205] Tried to move back new op lower bound from 7334449303018995712 to 7334449302705188864. Current Snapshot: MvccSnapshot[applied={T|T < 7334449303018995712}]
I20260928 23:10:36.873931 3882 mvcc.cc:205] Tried to move back new op lower bound from 7334449303018995712 to 7334449302705188864. Current Snapshot: MvccSnapshot[applied={T|T < 7334449303018995712 or (T in {7334449303018995712})}]
I20260928 23:10:36.875476 3403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:36.875568 3403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 23:10:36.876298 3403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 12169f5777b24cb88767eaba616a9921: Sending DeleteTablet for 3 replicas of tablet 4261dd41cf7a4ba9a1671c5d6b64fe82
I20260928 23:10:36.876601 3788 tablet_service.cc:1514] Processing DeleteTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:36 UTC) from {username='slave'} at 127.0.0.1:55714
I20260928 23:10:36.876704 3657 tablet_service.cc:1514] Processing DeleteTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:36 UTC) from {username='slave'} at 127.0.0.1:37708
I20260928 23:10:36.876729 3526 tablet_service.cc:1514] Processing DeleteTablet for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 23:10:36 UTC) from {username='slave'} at 127.0.0.1:52552
I20260928 23:10:36.876950 3893 tablet_replica.cc:330] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1: stopping tablet replica
I20260928 23:10:36.876992 3891 tablet_replica.cc:330] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: stopping tablet replica
I20260928 23:10:36.877053 3893 raft_consensus.cc:2229] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:36.877097 3891 raft_consensus.cc:2229] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 1 LEADER]: Raft consensus shutting down.
I20260928 23:10:36.877153 3893 raft_consensus.cc:2258] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P a85b2f780ca44d2e98a5398f8157eaf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:36.877197 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3464
I20260928 23:10:36.877208 3891 raft_consensus.cc:2258] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:36.877293 3892 tablet_replica.cc:330] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: stopping tablet replica
I20260928 23:10:36.877385 3892 raft_consensus.cc:2229] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 23:10:36.877470 3892 raft_consensus.cc:2258] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 23:10:36.877638 3891 ts_tablet_manager.cc:1905] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:36.878013 3892 ts_tablet_manager.cc:1905] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 23:10:36.879511 3892 ts_tablet_manager.cc:1918] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:36.879571 3892 log.cc:1196] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-1/wal/wals/4261dd41cf7a4ba9a1671c5d6b64fe82
I20260928 23:10:36.879580 3891 ts_tablet_manager.cc:1918] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 23:10:36.879643 3891 log.cc:1196] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Deleting WAL directory at /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790636798761935-16209-0/minicluster-data/ts-2/wal/wals/4261dd41cf7a4ba9a1671c5d6b64fe82
I20260928 23:10:36.879817 3892 ts_tablet_manager.cc:1939] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P afc868b172e148cbaeeaed45813c6a17: Deleting consensus metadata
I20260928 23:10:36.879886 3891 ts_tablet_manager.cc:1939] T 4261dd41cf7a4ba9a1671c5d6b64fe82 P bd6973eee85541c38945c06dbfc9daf4: Deleting consensus metadata
I20260928 23:10:36.880227 3390 catalog_manager.cc:4841] TS afc868b172e148cbaeeaed45813c6a17 (127.15.212.66:44571): tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 (table test-table [id=ddaafc8e73f140959fc4baf9c1fcd856]) successfully deleted
I20260928 23:10:36.880399 3391 catalog_manager.cc:4841] TS bd6973eee85541c38945c06dbfc9daf4 (127.15.212.67:39631): tablet 4261dd41cf7a4ba9a1671c5d6b64fe82 (table test-table [id=ddaafc8e73f140959fc4baf9c1fcd856]) successfully deleted
W20260928 23:10:36.885524 3389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.212.65:33429 (error 108)
I20260928 23:10:36.885725 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3595
I20260928 23:10:36.891204 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3726
I20260928 23:10:36.891362 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:36.891213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.65:33429 (local address 127.0.0.1:52564)
0928 23:10:36.891276 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 23:10:36.891322 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.65:33429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 23:10:36.891469 3389 catalog_manager.cc:4584] TS a85b2f780ca44d2e98a5398f8157eaf1 (127.15.212.65:33429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4261dd41cf7a4ba9a1671c5d6b64fe82: Network error: Client connection negotiation failed: client connection to 127.15.212.65:33429: connect: Connection refused (error 111)
I20260928 23:10:36.895620 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3373
2026-09-28T23:10:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (951 ms)
[----------] 6 tests from EncryptionPolicyTest (8607 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:36 dist-test-slave-x2t8 krb5kdc[3897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:36 dist-test-slave-x2t8 krb5kdc[3897](info): set up 2 sockets
Sep 28 23:10:36 dist-test-slave-x2t8 krb5kdc[3897](info): commencing operation
krb5kdc: starting...
W20260928 23:10:37.928184 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[3897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:38.047643 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:38.129035 3910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:38.129297 3910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:38.129333 3910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:38.131006 3910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:38.131072 3910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:38.131095 3910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:38.131112 3910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:38.131137 3910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:38.133121 3910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:38.133337 3910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:38.133505 3910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:38.138466 3915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:38.138535 3918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:38.138475 3916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:38.138757 3910 server_base.cc:1034] running on GCE node
I20260928 23:10:38.138931 3910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:38.139205 3910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:38.140344 3910 hybrid_clock.cc:648] HybridClock initialized: now 1790637038140333 us; error 30 us; skew 500 ppm
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[3897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:38.142700 3910 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:38.143162 3910 webserver.cc:460] Webserver started at http://127.0.0.1:44925/ using document root <none> and password file <none>
I20260928 23:10:38.143385 3910 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:38.143428 3910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:38.143575 3910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:38.144675 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "cf04e9efdb484e32b15311096ddd163d"
format_stamp: "Formatted at 2026-09-28 23:10:38 on dist-test-slave-x2t8"
server_key: "97f4ab74b1db90c9f06850deac1539a5"
server_key_iv: "d737a7c302da08eecad49a43804e805c"
server_key_version: "encryptionkey@0"
I20260928 23:10:38.145043 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "cf04e9efdb484e32b15311096ddd163d"
format_stamp: "Formatted at 2026-09-28 23:10:38 on dist-test-slave-x2t8"
server_key: "97f4ab74b1db90c9f06850deac1539a5"
server_key_iv: "d737a7c302da08eecad49a43804e805c"
server_key_version: "encryptionkey@0"
I20260928 23:10:38.146347 3910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:38.147061 3925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:38.147217 3910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:38.147320 3910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "cf04e9efdb484e32b15311096ddd163d"
format_stamp: "Formatted at 2026-09-28 23:10:38 on dist-test-slave-x2t8"
server_key: "97f4ab74b1db90c9f06850deac1539a5"
server_key_iv: "d737a7c302da08eecad49a43804e805c"
server_key_version: "encryptionkey@0"
I20260928 23:10:38.147389 3910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:38.161689 3910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:38.163539 3910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:38.163694 3910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:38.167603 3910 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36625
I20260928 23:10:38.167634 3977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36625 every 8 connection(s)
I20260928 23:10:38.167974 3910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:38.168910 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:38.171028 3978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d: Bootstrap starting.
I20260928 23:10:38.171564 3978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:38.171869 3978 log.cc:824] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:38.172353 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 3910
I20260928 23:10:38.172489 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:38.172638 3978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d: No bootstrap required, opened a new log
I20260928 23:10:38.172715 16209 external_mini_cluster.cc:1428] Setting key bdde815e9bf1bae3da427af4863f138f
I20260928 23:10:38.175002 3978 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260928 23:10:38.175135 3978 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:38.175158 3978 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf04e9efdb484e32b15311096ddd163d, State: Initialized, Role: FOLLOWER
I20260928 23:10:38.175276 3978 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260928 23:10:38.175346 3978 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:38.175369 3978 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:38.175431 3978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:38.175951 3978 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260928 23:10:38.176038 3978 leader_election.cc:302] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf04e9efdb484e32b15311096ddd163d; no voters:
I20260928 23:10:38.176206 3978 leader_election.cc:288] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:38.176244 3983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:38.176359 3983 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [term 1 LEADER]: Becoming Leader. State: Replica: cf04e9efdb484e32b15311096ddd163d, State: Running, Role: LEADER
I20260928 23:10:38.176493 3978 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:38.176477 3983 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260928 23:10:38.176782 3984 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf04e9efdb484e32b15311096ddd163d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } } }
I20260928 23:10:38.176980 3984 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:38.176927 3985 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf04e9efdb484e32b15311096ddd163d. Latest consensus state: current_term: 1 leader_uuid: "cf04e9efdb484e32b15311096ddd163d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf04e9efdb484e32b15311096ddd163d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } } }
I20260928 23:10:38.177439 3985 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:38.177623 3992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:38.178102 3992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[3897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 23:10:38.180794 3992 catalog_manager.cc:1347] Generated new cluster ID: 73c7623214594fecb2dc003929a36657
I20260928 23:10:38.180843 3992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:38.184885 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:38.173202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47351 (local address 127.0.0.1:36625)
0928 23:10:38.173391 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:38.173398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:38.173422 (+ 24us) server_negotiation.cc:407] Connection header received
0928 23:10:38.173936 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:38.173943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:38.173986 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:38.174096 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:38.174894 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:38.176231 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:38.177368 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:38.177988 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:38.178093 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:38.181193 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:38.181197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:38.181197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:38.181207 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:38.182993 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:38.183430 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:38.183433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:38.183433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:38.183525 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:38.183732 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:38.183733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:38.183733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:38.183992 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:38.184141 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:38.184610 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 23:10:38.184698 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260928 23:10:38.185699 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:38.194162 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:38.187179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55231 (local address 127.0.0.1:36625)
0928 23:10:38.187232 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:38.187234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:38.187310 (+ 76us) server_negotiation.cc:407] Connection header received
0928 23:10:38.187382 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:38.187383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:38.187390 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:38.187415 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:38.187972 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:38.188711 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:38.189738 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:38.190043 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:38.190096 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:38.191408 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:38.191409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:38.191409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:38.191416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:38.192890 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:38.193309 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:38.193310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:38.193311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:38.193361 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:38.193584 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:38.193584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:38.193584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:38.193687 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:38.193767 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:38.193946 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 23:10:38.193999 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:10:38.196389 3992 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:38.196887 3992 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:38.200700 3992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf04e9efdb484e32b15311096ddd163d: Generated new TSK 0
I20260928 23:10:38.200855 3992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:38.204262 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 3910
W20260928 23:10:38.208774 3895 connection.cc:535] client connection to 127.0.0.1:36625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36625): Connection reset by peer (error 104)
2026-09-28T23:10:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[4009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[4009](info): set up 2 sockets
Sep 28 23:10:38 dist-test-slave-x2t8 krb5kdc[4009](info): commencing operation
krb5kdc: starting...
W20260928 23:10:39.240916 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:39 dist-test-slave-x2t8 krb5kdc[4009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:39.351490 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:39.429765 4022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:39.429958 4022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:39.429980 4022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:39.431099 4022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:39.431142 4022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:39.431156 4022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:39.431169 4022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:39.431181 4022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:39.432627 4022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:39.432868 4022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:39.433101 4022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:39.437697 4030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:39.437745 4028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:39.437736 4027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:39.437999 4022 server_base.cc:1034] running on GCE node
I20260928 23:10:39.438262 4022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:39.438488 4022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:39.439652 4022 hybrid_clock.cc:648] HybridClock initialized: now 1790637039439645 us; error 39 us; skew 500 ppm
Sep 28 23:10:39 dist-test-slave-x2t8 krb5kdc[4009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637039, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:39.442021 4022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:39.442373 4022 webserver.cc:460] Webserver started at http://127.0.0.1:37055/ using document root <none> and password file <none>
I20260928 23:10:39.442538 4022 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:39.442576 4022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:39.442658 4022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:39.443583 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "8e694d9885b54671ab785dc07800a3a1"
format_stamp: "Formatted at 2026-09-28 23:10:39 on dist-test-slave-x2t8"
server_key: "5968ecef5c427d321fef67c77eb5de80"
server_key_iv: "a5d64245c770eabb7573fc534fec4c84"
server_key_version: "encryptionkey@0"
I20260928 23:10:39.443868 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "8e694d9885b54671ab785dc07800a3a1"
format_stamp: "Formatted at 2026-09-28 23:10:39 on dist-test-slave-x2t8"
server_key: "5968ecef5c427d321fef67c77eb5de80"
server_key_iv: "a5d64245c770eabb7573fc534fec4c84"
server_key_version: "encryptionkey@0"
I20260928 23:10:39.445128 4022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:39.445762 4037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:39.445935 4022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:39.446027 4022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "8e694d9885b54671ab785dc07800a3a1"
format_stamp: "Formatted at 2026-09-28 23:10:39 on dist-test-slave-x2t8"
server_key: "5968ecef5c427d321fef67c77eb5de80"
server_key_iv: "a5d64245c770eabb7573fc534fec4c84"
server_key_version: "encryptionkey@0"
I20260928 23:10:39.446092 4022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:39.457454 4022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:39.459612 4022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:39.459772 4022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:39.463515 4022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46071
I20260928 23:10:39.463577 4089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46071 every 8 connection(s)
I20260928 23:10:39.463891 4022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:39.464728 4090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:39.465389 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4022
I20260928 23:10:39.465517 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:39.465727 16209 external_mini_cluster.cc:1428] Setting key 7342c6c57668571835c54ded549ff4aa
I20260928 23:10:39.467391 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: Bootstrap starting.
I20260928 23:10:39.467960 4090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:39.468252 4090 log.cc:824] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:39.468899 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: No bootstrap required, opened a new log
I20260928 23:10:39.470734 4090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260928 23:10:39.470890 4090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:39.470958 4090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e694d9885b54671ab785dc07800a3a1, State: Initialized, Role: FOLLOWER
I20260928 23:10:39.471144 4090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260928 23:10:39.471233 4090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:39.471283 4090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:39.471338 4090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:39.471429 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:39.466194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59069 (local address 127.0.0.1:46071)
0928 23:10:39.466427 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:39.466435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:39.466463 (+ 28us) server_negotiation.cc:407] Connection header received
0928 23:10:39.467120 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:39.467128 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:39.467187 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:39.467326 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:39.467527 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:39.468579 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:39.469398 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:39.469839 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:39.469967 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:39.470678 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:39.470688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:39.470689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:39.470837 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:39.470864 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:39.470874 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:39.470968 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:39.471155 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 23:10:39.471182 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":65,"threads_started":1}
I20260928 23:10:39.471836 4090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260928 23:10:39.471949 4090 leader_election.cc:302] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e694d9885b54671ab785dc07800a3a1; no voters:
I20260928 23:10:39.472131 4090 leader_election.cc:288] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:39.472159 4095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:39.472265 4095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [term 1 LEADER]: Becoming Leader. State: Replica: 8e694d9885b54671ab785dc07800a3a1, State: Running, Role: LEADER
I20260928 23:10:39.472281 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:39.472363 4090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:39.472400 4095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } }
I20260928 23:10:39.472709 4095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e694d9885b54671ab785dc07800a3a1. Latest consensus state: current_term: 1 leader_uuid: "8e694d9885b54671ab785dc07800a3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } } }
I20260928 23:10:39.472781 4095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:39.472918 4096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e694d9885b54671ab785dc07800a3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e694d9885b54671ab785dc07800a3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46071 } } }
I20260928 23:10:39.472976 4096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1 [sys.catalog]: This master's current role is: LEADER
W20260928 23:10:39.474071 4115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:10:39.474154 4115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:10:39.474206 4101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:39.474341 4101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:39.475787 4101 catalog_manager.cc:1347] Generated new cluster ID: 963626ae9b9c42ada6a100d1dda63445
I20260928 23:10:39.475831 4101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:39.476984 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:39.473748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58517 (local address 127.0.0.1:46071)
0928 23:10:39.473780 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:39.473782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:39.473982 (+ 200us) server_negotiation.cc:407] Connection header received
0928 23:10:39.474048 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:39.474049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:39.474058 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:39.474109 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:39.474246 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:39.474689 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:39.475746 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:39.475969 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:39.476109 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:39.476767 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:39.476768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:39.476768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:39.476773 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:39.476784 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:39.476788 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:39.476840 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:39.476909 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 23:10:39.476918 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:10:39.489830 4101 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:39.490298 4101 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:39.494333 4101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e694d9885b54671ab785dc07800a3a1: Generated new TSK 0
I20260928 23:10:39.494496 4101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:39.498821 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4022
W20260928 23:10:39.502789 4007 connection.cc:535] client connection to 127.0.0.1:46071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46071): Connection reset by peer (error 104)
2026-09-28T23:10:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:39 dist-test-slave-x2t8 krb5kdc[4121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:39 dist-test-slave-x2t8 krb5kdc[4121](info): set up 2 sockets
Sep 28 23:10:39 dist-test-slave-x2t8 krb5kdc[4121](info): commencing operation
krb5kdc: starting...
W20260928 23:10:40.536381 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:40 dist-test-slave-x2t8 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:40.650580 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:40.727952 4134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:40.728171 4134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:40.728206 4134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:40.729360 4134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:40.729429 4134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:40.729451 4134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:40.729472 4134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:40.729501 4134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:40.730916 4134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:40.731178 4134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:40.731400 4134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:10:40.736054 4134 server_base.cc:1034] running on GCE node
W20260928 23:10:40.736002 4139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:40.735991 4142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:40.735968 4140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:40.736513 4134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:40.736785 4134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:40.737926 4134 hybrid_clock.cc:648] HybridClock initialized: now 1790637040737916 us; error 26 us; skew 500 ppm
Sep 28 23:10:40 dist-test-slave-x2t8 krb5kdc[4121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:40.740259 4134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:40.740800 4134 webserver.cc:460] Webserver started at http://127.0.0.1:45655/ using document root <none> and password file <none>
I20260928 23:10:40.741020 4134 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:40.741086 4134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:40.741214 4134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:40.742177 4134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "d87b49b51f5b4aeaac358256d67e90dd"
format_stamp: "Formatted at 2026-09-28 23:10:40 on dist-test-slave-x2t8"
server_key: "cef380a98a4760322e9193a169f6a087"
server_key_iv: "77f884fe6f71e4f568d834a7189cda47"
server_key_version: "encryptionkey@0"
I20260928 23:10:40.742530 4134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "d87b49b51f5b4aeaac358256d67e90dd"
format_stamp: "Formatted at 2026-09-28 23:10:40 on dist-test-slave-x2t8"
server_key: "cef380a98a4760322e9193a169f6a087"
server_key_iv: "77f884fe6f71e4f568d834a7189cda47"
server_key_version: "encryptionkey@0"
I20260928 23:10:40.743757 4134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:40.744506 4149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:40.744674 4134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:40.744765 4134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "d87b49b51f5b4aeaac358256d67e90dd"
format_stamp: "Formatted at 2026-09-28 23:10:40 on dist-test-slave-x2t8"
server_key: "cef380a98a4760322e9193a169f6a087"
server_key_iv: "77f884fe6f71e4f568d834a7189cda47"
server_key_version: "encryptionkey@0"
I20260928 23:10:40.744848 4134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:40.747843 4134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:40.749743 4134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:40.749895 4134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:40.753816 4134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33421
I20260928 23:10:40.753845 4201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33421 every 8 connection(s)
I20260928 23:10:40.754144 4134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:40.754599 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4134
I20260928 23:10:40.754727 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:40.754935 16209 external_mini_cluster.cc:1428] Setting key e4d9aa83a06d4a1804bbb98b43dc8aad
I20260928 23:10:40.754995 4202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:40.757411 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:40.755391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43811 (local address 127.0.0.1:33421)
0928 23:10:40.755592 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:40.755600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:40.755633 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:40.756156 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:40.756164 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:40.756198 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:40.756344 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:40.757014 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:40.757017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:40.757017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:40.757022 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:40.757061 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:40.757075 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:40.757153 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:40.757185 (+ 32us) server_negotiation.cc:299] Negotiation successful
0928 23:10:40.757222 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260928 23:10:40.758325 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:40.758378 4202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd: Bootstrap starting.
I20260928 23:10:40.758939 4202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:40.759248 4202 log.cc:824] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:40.759970 4202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd: No bootstrap required, opened a new log
I20260928 23:10:40.760744 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:40.759652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54263 (local address 127.0.0.1:33421)
0928 23:10:40.759687 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:40.759689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:40.759806 (+ 117us) server_negotiation.cc:407] Connection header received
0928 23:10:40.759845 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:40.759846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:40.759850 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:40.759917 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:40.760550 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:40.760551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:40.760551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:40.760555 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:40.760568 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:40.760572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:40.760639 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:40.760672 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 23:10:40.760684 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:40.762002 4202 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260928 23:10:40.762154 4202 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:40.762199 4202 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87b49b51f5b4aeaac358256d67e90dd, State: Initialized, Role: FOLLOWER
I20260928 23:10:40.762315 4202 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260928 23:10:40.762401 4202 raft_consensus.cc:388] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:40.762444 4202 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:40.762492 4202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:40.762972 4202 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260928 23:10:40.763089 4202 leader_election.cc:302] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d87b49b51f5b4aeaac358256d67e90dd; no voters:
I20260928 23:10:40.763260 4202 leader_election.cc:288] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:40.763301 4212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:40.763482 4202 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:40.763552 4212 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [term 1 LEADER]: Becoming Leader. State: Replica: d87b49b51f5b4aeaac358256d67e90dd, State: Running, Role: LEADER
I20260928 23:10:40.763680 4212 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } }
I20260928 23:10:40.764024 4213 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d87b49b51f5b4aeaac358256d67e90dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260928 23:10:40.764113 4213 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:40.764261 4213 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87b49b51f5b4aeaac358256d67e90dd. Latest consensus state: current_term: 1 leader_uuid: "d87b49b51f5b4aeaac358256d67e90dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b49b51f5b4aeaac358256d67e90dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33421 } } }
I20260928 23:10:40.764333 4213 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:40.764349 4221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:40.764860 4221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:40.766242 4221 catalog_manager.cc:1347] Generated new cluster ID: bef80d3ee93a469596fbb03d93179dd5
I20260928 23:10:40.766294 4221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:40.778071 4221 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:40.778215 4221 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:40.785424 4221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d87b49b51f5b4aeaac358256d67e90dd: Generated new TSK 0
I20260928 23:10:40.785588 4221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:40.787819 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4134
2026-09-28T23:10:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:40 dist-test-slave-x2t8 krb5kdc[4233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:40 dist-test-slave-x2t8 krb5kdc[4233](info): set up 2 sockets
Sep 28 23:10:40 dist-test-slave-x2t8 krb5kdc[4233](info): commencing operation
krb5kdc: starting...
W20260928 23:10:41.824319 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:41 dist-test-slave-x2t8 krb5kdc[4233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:41.935503 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44549
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:42.012414 4246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:42.012635 4246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:42.012657 4246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:42.013808 4246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:42.013851 4246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:42.013865 4246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:42.013878 4246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:42.013939 4246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:42.015336 4246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:42.015578 4246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:42.015799 4246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:42.020488 4254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:42.020520 4252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:42.020622 4246 server_base.cc:1034] running on GCE node
W20260928 23:10:42.020488 4251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:42.020989 4246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:42.021271 4246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:42.022432 4246 hybrid_clock.cc:648] HybridClock initialized: now 1790637042022415 us; error 39 us; skew 500 ppm
Sep 28 23:10:42 dist-test-slave-x2t8 krb5kdc[4233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:42.024838 4246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:42.025192 4246 webserver.cc:460] Webserver started at http://127.0.0.1:43371/ using document root <none> and password file <none>
I20260928 23:10:42.025377 4246 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:42.025416 4246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:42.025513 4246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:42.026448 4246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64"
format_stamp: "Formatted at 2026-09-28 23:10:42 on dist-test-slave-x2t8"
server_key: "14fdff1de41c9a6de88174c0b28bf5c8"
server_key_iv: "a265f9d983e0de262eee5031e74c6840"
server_key_version: "encryptionkey@0"
I20260928 23:10:42.026731 4246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64"
format_stamp: "Formatted at 2026-09-28 23:10:42 on dist-test-slave-x2t8"
server_key: "14fdff1de41c9a6de88174c0b28bf5c8"
server_key_iv: "a265f9d983e0de262eee5031e74c6840"
server_key_version: "encryptionkey@0"
I20260928 23:10:42.027880 4246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:42.028538 4261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:42.028707 4246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:42.028807 4246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64"
format_stamp: "Formatted at 2026-09-28 23:10:42 on dist-test-slave-x2t8"
server_key: "14fdff1de41c9a6de88174c0b28bf5c8"
server_key_iv: "a265f9d983e0de262eee5031e74c6840"
server_key_version: "encryptionkey@0"
I20260928 23:10:42.028894 4246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:42.042420 4246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:42.044574 4246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:42.044778 4246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:42.049154 4246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44549
I20260928 23:10:42.049213 4313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44549 every 8 connection(s)
I20260928 23:10:42.049547 4246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:42.050546 4314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:42.052742 4314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64: Bootstrap starting.
I20260928 23:10:42.053267 4314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:42.053536 4314 log.cc:824] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:42.054171 4314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64: No bootstrap required, opened a new log
I20260928 23:10:42.055665 4314 raft_consensus.cc:348] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260928 23:10:42.055788 4314 raft_consensus.cc:374] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:42.055831 4314 raft_consensus.cc:729] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebcd33d6da44dbdb7f8a0d05a7d5a64, State: Initialized, Role: FOLLOWER
I20260928 23:10:42.056016 4314 consensus_queue.cc:260] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260928 23:10:42.056077 4314 raft_consensus.cc:388] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:42.056110 4314 raft_consensus.cc:482] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:42.056173 4314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:42.056754 4314 raft_consensus.cc:504] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260928 23:10:42.056856 4314 leader_election.cc:302] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cebcd33d6da44dbdb7f8a0d05a7d5a64; no voters:
I20260928 23:10:42.057056 4314 leader_election.cc:288] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:42.057106 4317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:42.057283 4317 raft_consensus.cc:686] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [term 1 LEADER]: Becoming Leader. State: Replica: cebcd33d6da44dbdb7f8a0d05a7d5a64, State: Running, Role: LEADER
I20260928 23:10:42.057305 4314 sys_catalog.cc:564] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:42.057394 4317 consensus_queue.cc:237] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } }
I20260928 23:10:42.057786 4319 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cebcd33d6da44dbdb7f8a0d05a7d5a64. Latest consensus state: current_term: 1 leader_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } } }
I20260928 23:10:42.057871 4319 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:42.057974 4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cebcd33d6da44dbdb7f8a0d05a7d5a64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44549 } } }
I20260928 23:10:42.058066 4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:42.058429 4326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:42.058835 4326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:42.059574 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4246
I20260928 23:10:42.059692 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:42.059923 16209 external_mini_cluster.cc:1428] Setting key 3ed7d537ce36b047c2ab5eea98a1dfe2
I20260928 23:10:42.060328 4326 catalog_manager.cc:1347] Generated new cluster ID: 89bc0248c71e44de97793224dbcbb764
I20260928 23:10:42.060403 4326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:10:42 dist-test-slave-x2t8 krb5kdc[4233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 23:10:42.074461 4326 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:42.075135 4326 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:42.075192 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:42.060372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50753 (local address 127.0.0.1:44549)
0928 23:10:42.064565 (+ 4193us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:42.064575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:42.064608 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:42.064726 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:42.064733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:42.064788 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:42.064943 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:42.065664 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:42.067160 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:42.068192 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:42.068820 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:42.068929 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:42.071631 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:42.071635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:42.071636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:42.071647 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:42.073583 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:42.073977 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:42.073980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:42.073981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:42.074096 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:42.074304 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:42.074305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:42.074306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:42.074586 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:42.074735 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:42.074831 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 23:10:42.074967 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4148,"thread_start_us":83,"threads_started":1}
I20260928 23:10:42.076081 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:42.080717 4326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cebcd33d6da44dbdb7f8a0d05a7d5a64: Generated new TSK 0
I20260928 23:10:42.080869 4326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:42.086864 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:42.080524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40391 (local address 127.0.0.1:44549)
0928 23:10:42.080561 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:42.080563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:42.080743 (+ 180us) server_negotiation.cc:407] Connection header received
0928 23:10:42.080837 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:42.080838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:42.080849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:42.080897 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:42.081455 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:42.082199 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:42.082869 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:42.083179 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:42.083244 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:42.084285 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:42.084286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:42.084287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:42.084293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:42.085841 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:42.086147 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:42.086148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:42.086149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:42.086203 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:42.086400 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:42.086400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:42.086401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:42.086517 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:42.086632 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:42.086690 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 23:10:42.086763 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:42.088402 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4246
2026-09-28T23:10:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:42 dist-test-slave-x2t8 krb5kdc[4345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:42 dist-test-slave-x2t8 krb5kdc[4345](info): set up 2 sockets
Sep 28 23:10:42 dist-test-slave-x2t8 krb5kdc[4345](info): commencing operation
krb5kdc: starting...
W20260928 23:10:43.120370 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:43 dist-test-slave-x2t8 krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:43.240310 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41301
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:43.319883 4358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:43.320163 4358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:43.320204 4358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:43.321408 4358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:43.321482 4358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:43.321506 4358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:43.321527 4358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:43.321560 4358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:43.323065 4358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:43.323367 4358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:43.323602 4358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:43.328802 4363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:43.328867 4358 server_base.cc:1034] running on GCE node
W20260928 23:10:43.328796 4366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:43.328807 4364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:43.329293 4358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:43.329640 4358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:43.330789 4358 hybrid_clock.cc:648] HybridClock initialized: now 1790637043330778 us; error 31 us; skew 500 ppm
Sep 28 23:10:43 dist-test-slave-x2t8 krb5kdc[4345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637043, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:43.333268 4358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:43.333743 4358 webserver.cc:460] Webserver started at http://127.0.0.1:38697/ using document root <none> and password file <none>
I20260928 23:10:43.333936 4358 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:43.333978 4358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:43.334079 4358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:43.335156 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "a66854257e0f4584838f8da56d3049db"
format_stamp: "Formatted at 2026-09-28 23:10:43 on dist-test-slave-x2t8"
server_key: "8adf306cc0bec559920c9e6f2b91d1cd"
server_key_iv: "a53b81c3ffd8535acddf8eeeb41c3267"
server_key_version: "encryptionkey@0"
I20260928 23:10:43.335448 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "a66854257e0f4584838f8da56d3049db"
format_stamp: "Formatted at 2026-09-28 23:10:43 on dist-test-slave-x2t8"
server_key: "8adf306cc0bec559920c9e6f2b91d1cd"
server_key_iv: "a53b81c3ffd8535acddf8eeeb41c3267"
server_key_version: "encryptionkey@0"
I20260928 23:10:43.336767 4358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:43.337471 4373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:43.337666 4358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:43.337739 4358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "a66854257e0f4584838f8da56d3049db"
format_stamp: "Formatted at 2026-09-28 23:10:43 on dist-test-slave-x2t8"
server_key: "8adf306cc0bec559920c9e6f2b91d1cd"
server_key_iv: "a53b81c3ffd8535acddf8eeeb41c3267"
server_key_version: "encryptionkey@0"
I20260928 23:10:43.337827 4358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:43.353497 4358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:43.355590 4358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:43.355737 4358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:43.359819 4358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41301
I20260928 23:10:43.359908 4425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41301 every 8 connection(s)
I20260928 23:10:43.360234 4358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:43.361127 4426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:43.363440 4426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: Bootstrap starting.
I20260928 23:10:43.364039 4426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:43.364333 4426 log.cc:824] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:43.364766 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4358
I20260928 23:10:43.364885 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:43.365092 16209 external_mini_cluster.cc:1428] Setting key a0f51a46ea94ef73b826b44501bbfbe7
I20260928 23:10:43.365206 4426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: No bootstrap required, opened a new log
I20260928 23:10:43.367847 4426 raft_consensus.cc:348] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } }
I20260928 23:10:43.368032 4426 raft_consensus.cc:374] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:43.368072 4426 raft_consensus.cc:729] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66854257e0f4584838f8da56d3049db, State: Initialized, Role: FOLLOWER
I20260928 23:10:43.368199 4426 consensus_queue.cc:260] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } }
I20260928 23:10:43.368278 4426 raft_consensus.cc:388] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:43.368307 4426 raft_consensus.cc:482] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:43.368346 4426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:43.369095 4426 raft_consensus.cc:504] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } }
I20260928 23:10:43.369197 4426 leader_election.cc:302] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a66854257e0f4584838f8da56d3049db; no voters:
I20260928 23:10:43.369436 4431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:43.369446 4426 leader_election.cc:288] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:43.369611 4431 raft_consensus.cc:686] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [term 1 LEADER]: Becoming Leader. State: Replica: a66854257e0f4584838f8da56d3049db, State: Running, Role: LEADER
I20260928 23:10:43.369688 4431 consensus_queue.cc:237] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } }
I20260928 23:10:43.369748 4426 sys_catalog.cc:564] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:43.369985 4431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a66854257e0f4584838f8da56d3049db. Latest consensus state: current_term: 1 leader_uuid: "a66854257e0f4584838f8da56d3049db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } } }
I20260928 23:10:43.370056 4431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:43.370193 4431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a66854257e0f4584838f8da56d3049db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66854257e0f4584838f8da56d3049db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41301 } } }
I20260928 23:10:43.370239 4431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:43.371079 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:43.365603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36771 (local address 127.0.0.1:41301)
0928 23:10:43.365773 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:43.365782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:43.365817 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:10:43.366513 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:43.366520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:43.366571 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:43.366687 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:43.366937 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:43.367775 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:43.368908 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:43.369452 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:43.369583 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:43.370338 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:43.370342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:43.370342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:43.370589 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:43.370635 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:43.370652 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:43.370709 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:43.370795 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 23:10:43.370872 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
W20260928 23:10:43.372104 4447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:10:43.372161 4447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:10:43.372185 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:43.372217 4435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:43.372377 4435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:43.373939 4435 catalog_manager.cc:1347] Generated new cluster ID: ff967510a08c4d1f8a7eff6e636f40dd
I20260928 23:10:43.374011 4435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:43.379971 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:43.373655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53303 (local address 127.0.0.1:41301)
0928 23:10:43.373691 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:43.373693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:43.376610 (+ 2917us) server_negotiation.cc:407] Connection header received
0928 23:10:43.376711 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:43.376712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:43.376724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:43.376783 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:43.376913 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:43.377627 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:43.378698 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:43.378993 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:43.379053 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:43.379742 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:43.379743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:43.379743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:43.379747 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:43.379762 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:43.379766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:43.379815 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:43.379877 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 23:10:43.379898 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:10:43.392676 4435 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:43.393306 4435 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:43.404173 4435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a66854257e0f4584838f8da56d3049db: Generated new TSK 0
I20260928 23:10:43.404389 4435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:43.406289 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4358
2026-09-28T23:10:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:43 dist-test-slave-x2t8 krb5kdc[4457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:43 dist-test-slave-x2t8 krb5kdc[4457](info): set up 2 sockets
Sep 28 23:10:43 dist-test-slave-x2t8 krb5kdc[4457](info): commencing operation
krb5kdc: starting...
W20260928 23:10:44.436367 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:44 dist-test-slave-x2t8 krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:44.550594 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:44.629078 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:44.629312 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:44.629366 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:44.630537 4470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:44.630611 4470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:44.630636 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:44.630663 4470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:44.630707 4470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:44.632185 4470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:44.632507 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:44.632728 4470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:44.637627 4476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:44.637666 4478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:44.637696 4475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:44.637831 4470 server_base.cc:1034] running on GCE node
I20260928 23:10:44.638101 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:44.638376 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:44.639549 4470 hybrid_clock.cc:648] HybridClock initialized: now 1790637044639545 us; error 36 us; skew 500 ppm
Sep 28 23:10:44 dist-test-slave-x2t8 krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:44.647135 4470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:44.647593 4470 webserver.cc:460] Webserver started at http://127.0.0.1:44581/ using document root <none> and password file <none>
I20260928 23:10:44.647809 4470 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:44.647883 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:44.648028 4470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:44.649104 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "34e394ddc5564686ad50232c0786d402"
format_stamp: "Formatted at 2026-09-28 23:10:44 on dist-test-slave-x2t8"
server_key: "e68482b7b7af3642cef0da905e9a1a02"
server_key_iv: "b35d2539d3d942a5605e0af891844f00"
server_key_version: "encryptionkey@0"
I20260928 23:10:44.649477 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "34e394ddc5564686ad50232c0786d402"
format_stamp: "Formatted at 2026-09-28 23:10:44 on dist-test-slave-x2t8"
server_key: "e68482b7b7af3642cef0da905e9a1a02"
server_key_iv: "b35d2539d3d942a5605e0af891844f00"
server_key_version: "encryptionkey@0"
I20260928 23:10:44.650818 4470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:44.651578 4485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:44.651759 4470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:44.651860 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "34e394ddc5564686ad50232c0786d402"
format_stamp: "Formatted at 2026-09-28 23:10:44 on dist-test-slave-x2t8"
server_key: "e68482b7b7af3642cef0da905e9a1a02"
server_key_iv: "b35d2539d3d942a5605e0af891844f00"
server_key_version: "encryptionkey@0"
I20260928 23:10:44.651928 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:44.655179 4470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:44.656980 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:44.657135 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:44.661005 4470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35103
I20260928 23:10:44.661046 4537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35103 every 8 connection(s)
I20260928 23:10:44.661363 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:44.662148 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:44.664167 4538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: Bootstrap starting.
I20260928 23:10:44.664727 4538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:44.664969 4538 log.cc:824] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:44.665096 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4470
I20260928 23:10:44.665236 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:44.665437 16209 external_mini_cluster.cc:1428] Setting key ccaea89d9d851c68e4daf0ba74b03028
I20260928 23:10:44.665684 4538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: No bootstrap required, opened a new log
I20260928 23:10:44.667464 4538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260928 23:10:44.667619 4538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:44.667662 4538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e394ddc5564686ad50232c0786d402, State: Initialized, Role: FOLLOWER
I20260928 23:10:44.667788 4538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260928 23:10:44.667886 4538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:44.667922 4538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:44.667977 4538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:44.668648 4542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:44.665864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44437 (local address 127.0.0.1:35103)
0928 23:10:44.666117 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:44.666131 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:44.666153 (+ 22us) server_negotiation.cc:407] Connection header received
0928 23:10:44.667452 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:44.667457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:44.667481 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:44.667618 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:44.668201 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:44.668205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:44.668205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:44.668209 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:44.668255 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:44.668266 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:44.668339 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:44.668409 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 23:10:44.668474 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":132,"threads_started":1}
I20260928 23:10:44.669063 4538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260928 23:10:44.669171 4538 leader_election.cc:302] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34e394ddc5564686ad50232c0786d402; no voters:
I20260928 23:10:44.669315 4538 leader_election.cc:288] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:44.669348 4543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:44.669526 4543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [term 1 LEADER]: Becoming Leader. State: Replica: 34e394ddc5564686ad50232c0786d402, State: Running, Role: LEADER
I20260928 23:10:44.669533 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:44.669539 4538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:44.669617 4543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } }
I20260928 23:10:44.669895 4543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34e394ddc5564686ad50232c0786d402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260928 23:10:44.669953 4543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:44.670077 4543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e394ddc5564686ad50232c0786d402. Latest consensus state: current_term: 1 leader_uuid: "34e394ddc5564686ad50232c0786d402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e394ddc5564686ad50232c0786d402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35103 } } }
I20260928 23:10:44.670118 4543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402 [sys.catalog]: This master's current role is: LEADER
W20260928 23:10:44.671268 4563 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:10:44.671324 4563 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:10:44.671419 4550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:44.671622 4550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:44.673287 4550 catalog_manager.cc:1347] Generated new cluster ID: abfbf5417d8c4f548b20aeb7836cfa26
I20260928 23:10:44.673345 4550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:44.673625 4542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:44.671219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60237 (local address 127.0.0.1:35103)
0928 23:10:44.671256 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:44.671258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:44.672648 (+ 1390us) server_negotiation.cc:407] Connection header received
0928 23:10:44.672722 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:44.672723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:44.672727 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:44.672772 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:44.673348 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:44.673349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:44.673350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:44.673354 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:44.673369 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:44.673373 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:44.673452 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:44.673540 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 23:10:44.673554 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:44.685941 4550 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:44.686096 4550 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:44.689510 4550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34e394ddc5564686ad50232c0786d402: Generated new TSK 0
I20260928 23:10:44.689677 4550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:44.698570 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4470
2026-09-28T23:10:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:44 dist-test-slave-x2t8 krb5kdc[4569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:44 dist-test-slave-x2t8 krb5kdc[4569](info): set up 2 sockets
Sep 28 23:10:44 dist-test-slave-x2t8 krb5kdc[4569](info): commencing operation
krb5kdc: starting...
W20260928 23:10:45.712360 16209 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:45 dist-test-slave-x2t8 krb5kdc[4569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:45.827466 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:45.906792 4582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:45.907021 4582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:45.907060 4582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:45.908172 4582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:45.908239 4582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:45.908262 4582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:45.908284 4582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:45.908311 4582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:45.909839 4582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:45.910115 4582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:45.910354 4582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:45.915171 4588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:45.915180 4587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:45.915180 4590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:45.915390 4582 server_base.cc:1034] running on GCE node
I20260928 23:10:45.915648 4582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:45.915921 4582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:45.917089 4582 hybrid_clock.cc:648] HybridClock initialized: now 1790637045917070 us; error 39 us; skew 500 ppm
Sep 28 23:10:45 dist-test-slave-x2t8 krb5kdc[4569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637045, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:45.919354 4582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:45.919750 4582 webserver.cc:460] Webserver started at http://127.0.0.1:35599/ using document root <none> and password file <none>
I20260928 23:10:45.919982 4582 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:45.920051 4582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:45.920202 4582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:45.921262 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "957458dc99194bd68635d8f7070910ab"
format_stamp: "Formatted at 2026-09-28 23:10:45 on dist-test-slave-x2t8"
server_key: "843bc4d4c7df3e1fde0aa1ed3d97cb04"
server_key_iv: "6eaac31b5717a948e13f32b9b5bf1769"
server_key_version: "encryptionkey@0"
I20260928 23:10:45.921636 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "957458dc99194bd68635d8f7070910ab"
format_stamp: "Formatted at 2026-09-28 23:10:45 on dist-test-slave-x2t8"
server_key: "843bc4d4c7df3e1fde0aa1ed3d97cb04"
server_key_iv: "6eaac31b5717a948e13f32b9b5bf1769"
server_key_version: "encryptionkey@0"
I20260928 23:10:45.922905 4582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:45.923650 4597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:45.923831 4582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:45.923930 4582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "957458dc99194bd68635d8f7070910ab"
format_stamp: "Formatted at 2026-09-28 23:10:45 on dist-test-slave-x2t8"
server_key: "843bc4d4c7df3e1fde0aa1ed3d97cb04"
server_key_iv: "6eaac31b5717a948e13f32b9b5bf1769"
server_key_version: "encryptionkey@0"
I20260928 23:10:45.924023 4582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:45.932251 4582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:45.933893 4582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:45.934070 4582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:45.938230 4582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46859
I20260928 23:10:45.938258 4649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46859 every 8 connection(s)
I20260928 23:10:45.938592 4582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:45.939414 4650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:45.941390 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4582
I20260928 23:10:45.941511 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:45.941738 16209 external_mini_cluster.cc:1428] Setting key ae11eefeedf51435f4208bc717bde12e
I20260928 23:10:45.941740 4650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab: Bootstrap starting.
I20260928 23:10:45.942371 4650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:45.942646 4650 log.cc:824] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:45.943248 4650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab: No bootstrap required, opened a new log
I20260928 23:10:45.944974 4650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260928 23:10:45.945122 4650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:45.945169 4650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957458dc99194bd68635d8f7070910ab, State: Initialized, Role: FOLLOWER
I20260928 23:10:45.945303 4650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260928 23:10:45.945389 4650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:45.945431 4650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:45.945479 4650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:45.946110 4650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260928 23:10:45.946215 4650 leader_election.cc:302] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 957458dc99194bd68635d8f7070910ab; no voters:
I20260928 23:10:45.946359 4650 leader_election.cc:288] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:45.946413 4655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:45.946597 4655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [term 1 LEADER]: Becoming Leader. State: Replica: 957458dc99194bd68635d8f7070910ab, State: Running, Role: LEADER
I20260928 23:10:45.946743 4655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260928 23:10:45.946887 4650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:45.947181 4656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "957458dc99194bd68635d8f7070910ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260928 23:10:45.947259 4656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:45.947427 4657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 957458dc99194bd68635d8f7070910ab. Latest consensus state: current_term: 1 leader_uuid: "957458dc99194bd68635d8f7070910ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "957458dc99194bd68635d8f7070910ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260928 23:10:45.947497 4657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:45.947518 4659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:45.947999 4659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 23:10:45 dist-test-slave-x2t8 krb5kdc[4569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 23:10:45.949616 4659 catalog_manager.cc:1347] Generated new cluster ID: e4eb27cf2b5d41fbb8c8a9292dd2f795
I20260928 23:10:45.949663 4659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:45.955564 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:45.942214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:49991 (local address 127.0.0.1:46859)
0928 23:10:45.942382 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:45.942391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:45.942419 (+ 28us) server_negotiation.cc:407] Connection header received
0928 23:10:45.942900 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:45.942906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:45.942952 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:45.943086 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:45.943711 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:45.944601 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:45.945504 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:45.946130 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:45.946245 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:45.949229 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:45.949233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:45.949234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:45.949244 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:45.953802 (+ 4558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:45.954266 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:45.954270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:45.954271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:45.954402 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:45.954666 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:45.954667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:45.954668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:45.954988 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:45.955148 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:45.955271 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 23:10:45.955370 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260928 23:10:45.956595 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:45.959769 4659 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:45.960220 4659 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:45.964864 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:45.957936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:32993 (local address 127.0.0.1:46859)
0928 23:10:45.958009 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:45.958011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:45.958129 (+ 118us) server_negotiation.cc:407] Connection header received
0928 23:10:45.958172 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:45.958173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:45.958183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:45.958243 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:45.958820 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:45.959324 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:45.960504 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:45.960838 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:45.960876 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:45.962195 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:45.962196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:45.962196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:45.962202 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:45.963620 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:45.964028 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:45.964029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:45.964030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:45.964078 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:45.964292 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:45.964293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:45.964293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:45.964409 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:45.964501 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:45.964722 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 23:10:45.964783 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 23:10:45.967335 4659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 957458dc99194bd68635d8f7070910ab: Generated new TSK 0
I20260928 23:10:45.967501 4659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:45.971581 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4582
W20260928 23:10:45.975638 4567 connection.cc:535] client connection to 127.0.0.1:46859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46859): Connection reset by peer (error 104)
2026-09-28T23:10:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:46 dist-test-slave-x2t8 krb5kdc[4681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:46 dist-test-slave-x2t8 krb5kdc[4681](info): set up 2 sockets
Sep 28 23:10:46 dist-test-slave-x2t8 krb5kdc[4681](info): commencing operation
krb5kdc: starting...
W20260928 23:10:47.000607 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:47 dist-test-slave-x2t8 krb5kdc[4681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:47.115526 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:47.192632 4694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:47.192878 4694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:47.192915 4694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:47.194056 4694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:47.194131 4694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:47.194155 4694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:47.194185 4694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:47.194211 4694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:47.195700 4694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:47.195973 4694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:47.196213 4694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:47.200911 4702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:47.200939 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:47.200982 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:47.201135 4694 server_base.cc:1034] running on GCE node
I20260928 23:10:47.201349 4694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:47.201577 4694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:47.202713 4694 hybrid_clock.cc:648] HybridClock initialized: now 1790637047202692 us; error 37 us; skew 500 ppm
Sep 28 23:10:47 dist-test-slave-x2t8 krb5kdc[4681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637047, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:47.205168 4694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:47.205601 4694 webserver.cc:460] Webserver started at http://127.0.0.1:41597/ using document root <none> and password file <none>
I20260928 23:10:47.205786 4694 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:47.205825 4694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:47.205910 4694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:47.206874 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "9fb23fa8399541b4a9c048ce4c58f8c4"
format_stamp: "Formatted at 2026-09-28 23:10:47 on dist-test-slave-x2t8"
server_key: "f4e874570bb043eed6f7a75fc2f491fd"
server_key_iv: "87588faaa9acac359de96c023b44e8bb"
server_key_version: "encryptionkey@0"
I20260928 23:10:47.207172 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "9fb23fa8399541b4a9c048ce4c58f8c4"
format_stamp: "Formatted at 2026-09-28 23:10:47 on dist-test-slave-x2t8"
server_key: "f4e874570bb043eed6f7a75fc2f491fd"
server_key_iv: "87588faaa9acac359de96c023b44e8bb"
server_key_version: "encryptionkey@0"
I20260928 23:10:47.208369 4694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 23:10:47.209182 4709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:47.209379 4694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:47.209468 4694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "9fb23fa8399541b4a9c048ce4c58f8c4"
format_stamp: "Formatted at 2026-09-28 23:10:47 on dist-test-slave-x2t8"
server_key: "f4e874570bb043eed6f7a75fc2f491fd"
server_key_iv: "87588faaa9acac359de96c023b44e8bb"
server_key_version: "encryptionkey@0"
I20260928 23:10:47.209527 4694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:47.239320 4694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:47.241667 4694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:47.241775 4694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:47.245476 4694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38795
I20260928 23:10:47.245553 4761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38795 every 8 connection(s)
I20260928 23:10:47.245880 4694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:47.246728 4762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:47.248952 4762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4: Bootstrap starting.
I20260928 23:10:47.249501 4762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:47.249725 4762 log.cc:824] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:47.249958 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4694
I20260928 23:10:47.250089 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:47.250339 16209 external_mini_cluster.cc:1428] Setting key dec25e7d219a69c4fcdd8d75e8debbd7
I20260928 23:10:47.250571 4762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4: No bootstrap required, opened a new log
I20260928 23:10:47.252558 4762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } }
I20260928 23:10:47.252708 4762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:47.252753 4762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fb23fa8399541b4a9c048ce4c58f8c4, State: Initialized, Role: FOLLOWER
I20260928 23:10:47.252915 4762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } }
I20260928 23:10:47.253012 4762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:47.253055 4762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:47.253101 4762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:47.253611 4762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } }
I20260928 23:10:47.253726 4762 leader_election.cc:302] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fb23fa8399541b4a9c048ce4c58f8c4; no voters:
I20260928 23:10:47.253892 4762 leader_election.cc:288] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:47.253913 4767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:47.254032 4767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [term 1 LEADER]: Becoming Leader. State: Replica: 9fb23fa8399541b4a9c048ce4c58f8c4, State: Running, Role: LEADER
I20260928 23:10:47.254109 4767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } }
I20260928 23:10:47.254163 4762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:47.254422 4769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } } }
I20260928 23:10:47.254523 4769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:47.254684 4768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fb23fa8399541b4a9c048ce4c58f8c4. Latest consensus state: current_term: 1 leader_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fb23fa8399541b4a9c048ce4c58f8c4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38795 } } }
I20260928 23:10:47.254773 4768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:47.254911 4772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:47.255466 4772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:47.257263 4772 catalog_manager.cc:1347] Generated new cluster ID: eb7ab4661bef4c9fb09056fb4e77971c
I20260928 23:10:47.257310 4772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:47.257315 4766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:47.250839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:45977 (local address 127.0.0.1:38795)
0928 23:10:47.251018 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:47.251027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:47.251060 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:47.251799 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:47.251805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:47.251857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:47.252000 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:47.252211 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:47.253599 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:47.254694 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:47.255522 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:47.255647 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:47.256522 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:47.256525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:47.256526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:47.256723 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:47.256761 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:47.256775 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:47.256860 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:47.257041 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 23:10:47.257069 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260928 23:10:47.258359 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:47.263273 4766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:47.259775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48907 (local address 127.0.0.1:38795)
0928 23:10:47.259836 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:47.259838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:47.260017 (+ 179us) server_negotiation.cc:407] Connection header received
0928 23:10:47.260073 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:47.260074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:47.260083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:47.260129 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:47.260260 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:47.260948 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:47.261969 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:47.262257 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:47.262315 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:47.263023 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:47.263024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:47.263024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:47.263028 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:47.263041 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:47.263045 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:47.263096 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:47.263197 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 23:10:47.263205 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 23:10:47.277992 4772 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:47.278788 4772 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:47.282068 4772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fb23fa8399541b4a9c048ce4c58f8c4: Generated new TSK 0
I20260928 23:10:47.282236 4772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:47.288815 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4694
W20260928 23:10:47.292590 4679 connection.cc:535] client connection to 127.0.0.1:38795 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38795): Connection reset by peer (error 104)
2026-09-28T23:10:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:47 dist-test-slave-x2t8 krb5kdc[4793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:47 dist-test-slave-x2t8 krb5kdc[4793](info): set up 2 sockets
Sep 28 23:10:47 dist-test-slave-x2t8 krb5kdc[4793](info): commencing operation
krb5kdc: starting...
W20260928 23:10:48.316409 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:48 dist-test-slave-x2t8 krb5kdc[4793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:48.433705 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:48.513516 4806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:48.513753 4806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:48.513792 4806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:48.514955 4806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:48.515027 4806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:48.515054 4806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:48.515085 4806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:48.515110 4806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:48.516655 4806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:48.516933 4806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:48.517164 4806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:48.522006 4812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:48.522054 4811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:48.522084 4814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:48.522255 4806 server_base.cc:1034] running on GCE node
I20260928 23:10:48.522477 4806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:48.522765 4806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:48.523948 4806 hybrid_clock.cc:648] HybridClock initialized: now 1790637048523935 us; error 48 us; skew 500 ppm
Sep 28 23:10:48 dist-test-slave-x2t8 krb5kdc[4793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637048, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:48.526371 4806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:48.526834 4806 webserver.cc:460] Webserver started at http://127.0.0.1:34219/ using document root <none> and password file <none>
I20260928 23:10:48.527067 4806 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:48.527134 4806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:48.527282 4806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:48.528330 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "1ca19cb367b14d2bb01f0ba79989c703"
format_stamp: "Formatted at 2026-09-28 23:10:48 on dist-test-slave-x2t8"
server_key: "ebcaa75eb19f5e141e9fe6a740d89f7d"
server_key_iv: "0cc8b0347a5c919519ccdfff42e3ea27"
server_key_version: "encryptionkey@0"
I20260928 23:10:48.528764 4806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "1ca19cb367b14d2bb01f0ba79989c703"
format_stamp: "Formatted at 2026-09-28 23:10:48 on dist-test-slave-x2t8"
server_key: "ebcaa75eb19f5e141e9fe6a740d89f7d"
server_key_iv: "0cc8b0347a5c919519ccdfff42e3ea27"
server_key_version: "encryptionkey@0"
I20260928 23:10:48.530066 4806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:48.530792 4821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:48.530967 4806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:48.531059 4806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "1ca19cb367b14d2bb01f0ba79989c703"
format_stamp: "Formatted at 2026-09-28 23:10:48 on dist-test-slave-x2t8"
server_key: "ebcaa75eb19f5e141e9fe6a740d89f7d"
server_key_iv: "0cc8b0347a5c919519ccdfff42e3ea27"
server_key_version: "encryptionkey@0"
I20260928 23:10:48.531147 4806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:48.534364 4806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:48.536139 4806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:48.536290 4806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:48.540143 4806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39989
I20260928 23:10:48.540171 4873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39989 every 8 connection(s)
I20260928 23:10:48.540580 4806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:48.541366 4874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:48.543430 4874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703: Bootstrap starting.
I20260928 23:10:48.543922 4874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:48.544140 4874 log.cc:824] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:48.544767 4874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703: No bootstrap required, opened a new log
I20260928 23:10:48.546219 4874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } }
I20260928 23:10:48.546343 4874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:48.546365 4874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca19cb367b14d2bb01f0ba79989c703, State: Initialized, Role: FOLLOWER
I20260928 23:10:48.546495 4874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } }
I20260928 23:10:48.546572 4874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:48.546594 4874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:48.546622 4874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:48.547058 4874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } }
I20260928 23:10:48.547135 4874 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca19cb367b14d2bb01f0ba79989c703; no voters:
I20260928 23:10:48.547250 4874 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:48.547322 4877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:48.547465 4877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca19cb367b14d2bb01f0ba79989c703, State: Running, Role: LEADER
I20260928 23:10:48.547505 4874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:48.547551 4877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } }
I20260928 23:10:48.548069 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4806
I20260928 23:10:48.548046 4878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca19cb367b14d2bb01f0ba79989c703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } } }
I20260928 23:10:48.548125 4878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:48.548189 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:48.548305 4879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca19cb367b14d2bb01f0ba79989c703. Latest consensus state: current_term: 1 leader_uuid: "1ca19cb367b14d2bb01f0ba79989c703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca19cb367b14d2bb01f0ba79989c703" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39989 } } }
I20260928 23:10:48.548380 4879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:48.548429 16209 external_mini_cluster.cc:1428] Setting key c1e08d749bb5743e34b5cc8d6af2b557
I20260928 23:10:48.548872 4886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:48.549587 4886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:48.550804 4895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:48.549072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:51753 (local address 127.0.0.1:39989)
0928 23:10:48.549237 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:48.549246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:48.549279 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:48.549398 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:48.549406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:48.549436 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:48.549584 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:48.550339 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:48.550343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:48.550343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:48.550349 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:48.550388 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:48.550402 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:48.550477 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:48.550557 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 23:10:48.550600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260928 23:10:48.551281 4886 catalog_manager.cc:1347] Generated new cluster ID: 7c3aa4932b3a4f1db3f93197f5dd8125
I20260928 23:10:48.551324 4886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:48.551651 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:48.557551 4895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:48.553097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:43535 (local address 127.0.0.1:39989)
0928 23:10:48.556512 (+ 3415us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:48.556514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:48.556523 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:10:48.556544 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:48.556545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:48.556550 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:48.556612 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:48.557220 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:48.557221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:48.557222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:48.557226 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:48.557250 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:48.557255 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:48.557316 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:48.557379 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 23:10:48.557392 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3395}
I20260928 23:10:48.576866 4886 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:48.577096 4886 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:48.582782 4886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca19cb367b14d2bb01f0ba79989c703: Generated new TSK 0
I20260928 23:10:48.582975 4886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:48.591027 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4806
2026-09-28T23:10:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:48 dist-test-slave-x2t8 krb5kdc[4905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:48 dist-test-slave-x2t8 krb5kdc[4905](info): set up 2 sockets
Sep 28 23:10:48 dist-test-slave-x2t8 krb5kdc[4905](info): commencing operation
krb5kdc: starting...
W20260928 23:10:49.620638 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[4905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:49.731340 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:49.808383 4918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:49.808583 4918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:49.808605 4918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:49.809731 4918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:49.809772 4918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:49.809785 4918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:49.809823 4918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:49.809849 4918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:49.811249 4918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:49.811450 4918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:49.811635 4918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:49.816196 4924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:49.816236 4926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:49.816288 4918 server_base.cc:1034] running on GCE node
W20260928 23:10:49.816236 4923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:49.816673 4918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:49.816987 4918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:49.818145 4918 hybrid_clock.cc:648] HybridClock initialized: now 1790637049818142 us; error 36 us; skew 500 ppm
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[4905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637049, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:49.827236 4918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:49.827651 4918 webserver.cc:460] Webserver started at http://127.0.0.1:43761/ using document root <none> and password file <none>
I20260928 23:10:49.827836 4918 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:49.827874 4918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:49.827955 4918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:49.828970 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "c919d8011c374bf6bce636c44512d6ee"
format_stamp: "Formatted at 2026-09-28 23:10:49 on dist-test-slave-x2t8"
server_key: "a420efde913df4ce2e02f4ba4822d5aa"
server_key_iv: "48d00e6702b8cc04147e75b312241120"
server_key_version: "encryptionkey@0"
I20260928 23:10:49.829267 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "c919d8011c374bf6bce636c44512d6ee"
format_stamp: "Formatted at 2026-09-28 23:10:49 on dist-test-slave-x2t8"
server_key: "a420efde913df4ce2e02f4ba4822d5aa"
server_key_iv: "48d00e6702b8cc04147e75b312241120"
server_key_version: "encryptionkey@0"
I20260928 23:10:49.830454 4918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:10:49.831122 4933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:49.831314 4918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:49.831386 4918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "c919d8011c374bf6bce636c44512d6ee"
format_stamp: "Formatted at 2026-09-28 23:10:49 on dist-test-slave-x2t8"
server_key: "a420efde913df4ce2e02f4ba4822d5aa"
server_key_iv: "48d00e6702b8cc04147e75b312241120"
server_key_version: "encryptionkey@0"
I20260928 23:10:49.831439 4918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:49.845315 4918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:49.847119 4918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:49.847220 4918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:49.850903 4918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37219
I20260928 23:10:49.850952 4985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37219 every 8 connection(s)
I20260928 23:10:49.851285 4918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:49.852180 4986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:49.854405 4986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee: Bootstrap starting.
I20260928 23:10:49.854940 4986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:49.855165 4986 log.cc:824] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:49.855520 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 4918
I20260928 23:10:49.855661 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:49.855878 16209 external_mini_cluster.cc:1428] Setting key 8e0ac5f4bb17dee40428de906208ff80
I20260928 23:10:49.855883 4986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee: No bootstrap required, opened a new log
I20260928 23:10:49.857734 4986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } }
I20260928 23:10:49.857905 4986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:49.857945 4986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c919d8011c374bf6bce636c44512d6ee, State: Initialized, Role: FOLLOWER
I20260928 23:10:49.858080 4986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } }
I20260928 23:10:49.858165 4986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:49.858201 4986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:49.858250 4986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:49.859215 4986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } }
I20260928 23:10:49.859302 4986 leader_election.cc:302] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c919d8011c374bf6bce636c44512d6ee; no voters:
I20260928 23:10:49.859620 4986 leader_election.cc:288] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:49.859637 4991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:49.859789 4991 raft_consensus.cc:686] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [term 1 LEADER]: Becoming Leader. State: Replica: c919d8011c374bf6bce636c44512d6ee, State: Running, Role: LEADER
I20260928 23:10:49.859918 4986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:49.859930 4991 consensus_queue.cc:237] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } }
I20260928 23:10:49.860337 4992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c919d8011c374bf6bce636c44512d6ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } } }
I20260928 23:10:49.860352 4993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader c919d8011c374bf6bce636c44512d6ee. Latest consensus state: current_term: 1 leader_uuid: "c919d8011c374bf6bce636c44512d6ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c919d8011c374bf6bce636c44512d6ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37219 } } }
I20260928 23:10:49.860507 4992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:49.860714 4993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:49.860747 4999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:49.861320 4999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:49.863102 4999 catalog_manager.cc:1347] Generated new cluster ID: aae867cc9e954a639152fa807caf1026
I20260928 23:10:49.863188 4999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[4905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 23:10:49.867489 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:49.856365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:60857 (local address 127.0.0.1:37219)
0928 23:10:49.856551 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:49.856560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:49.856593 (+ 33us) server_negotiation.cc:407] Connection header received
0928 23:10:49.857719 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:49.857725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:49.857773 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:49.857882 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:49.858700 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:49.859671 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:49.860798 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:49.861580 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:49.861992 (+ 412us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:49.864335 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:49.864338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:49.864339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:49.864350 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:49.865913 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:49.866295 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:49.866298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:49.866299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:49.866412 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:49.866635 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:49.866637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:49.866637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:49.866861 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:49.867070 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:49.867154 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 23:10:49.867351 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260928 23:10:49.868330 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:49.876075 4999 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:49.876776 4999 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:49.878577 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:49.869673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48711 (local address 127.0.0.1:37219)
0928 23:10:49.869708 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:49.869710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:49.872645 (+ 2935us) server_negotiation.cc:407] Connection header received
0928 23:10:49.872717 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:49.872718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:49.872728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:49.872776 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:49.873285 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:49.873749 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:49.874478 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:49.874706 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:49.874775 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:49.876009 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:49.876010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:49.876011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:49.876018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:49.877419 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:49.877785 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:49.877786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:49.877787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:49.877839 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:49.878061 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:49.878062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:49.878063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:49.878167 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:49.878274 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:49.878396 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 23:10:49.878457 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:49.879994 4999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c919d8011c374bf6bce636c44512d6ee: Generated new TSK 0
I20260928 23:10:49.880151 4999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:49.883737 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 4918
W20260928 23:10:49.887655 4903 connection.cc:535] client connection to 127.0.0.1:37219 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37219): Connection reset by peer (error 104)
2026-09-28T23:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[5017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[5017](info): set up 2 sockets
Sep 28 23:10:49 dist-test-slave-x2t8 krb5kdc[5017](info): commencing operation
krb5kdc: starting...
W20260928 23:10:50.908654 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:50 dist-test-slave-x2t8 krb5kdc[5017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:51.021409 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:51.100948 5030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:51.101132 5030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:51.101154 5030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:51.102226 5030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:51.102259 5030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:51.102273 5030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:51.102286 5030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:51.102298 5030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:51.103660 5030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44833
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:51.103839 5030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:51.103986 5030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:51.108584 5036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:51.108698 5030 server_base.cc:1034] running on GCE node
W20260928 23:10:51.108597 5035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:51.108592 5038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:51.109006 5030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:51.109234 5030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:51.110415 5030 hybrid_clock.cc:648] HybridClock initialized: now 1790637051110396 us; error 34 us; skew 500 ppm
Sep 28 23:10:51 dist-test-slave-x2t8 krb5kdc[5017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637051, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:51.112884 5030 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:51.113281 5030 webserver.cc:460] Webserver started at http://127.0.0.1:40609/ using document root <none> and password file <none>
I20260928 23:10:51.113505 5030 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:51.113574 5030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:51.113718 5030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:51.114691 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278"
format_stamp: "Formatted at 2026-09-28 23:10:51 on dist-test-slave-x2t8"
server_key: "5e4364516de8e2b3a6c6fff12da536d6"
server_key_iv: "0a09c27ff77b3c064b921563352a5902"
server_key_version: "encryptionkey@0"
I20260928 23:10:51.115079 5030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278"
format_stamp: "Formatted at 2026-09-28 23:10:51 on dist-test-slave-x2t8"
server_key: "5e4364516de8e2b3a6c6fff12da536d6"
server_key_iv: "0a09c27ff77b3c064b921563352a5902"
server_key_version: "encryptionkey@0"
I20260928 23:10:51.116314 5030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:10:51.117043 5045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:51.117209 5030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:51.117311 5030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278"
format_stamp: "Formatted at 2026-09-28 23:10:51 on dist-test-slave-x2t8"
server_key: "5e4364516de8e2b3a6c6fff12da536d6"
server_key_iv: "0a09c27ff77b3c064b921563352a5902"
server_key_version: "encryptionkey@0"
I20260928 23:10:51.117394 5030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:51.143760 5030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:51.146059 5030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:51.146211 5030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:51.149961 5030 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44833
I20260928 23:10:51.150032 5097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44833 every 8 connection(s)
I20260928 23:10:51.150360 5030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:51.151149 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:51.153414 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278: Bootstrap starting.
I20260928 23:10:51.153967 5098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:51.154246 5098 log.cc:824] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:51.154923 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278: No bootstrap required, opened a new log
I20260928 23:10:51.155366 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5030
I20260928 23:10:51.155586 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:51.155797 16209 external_mini_cluster.cc:1428] Setting key 74694e7b47c2c8998cecd5db078f1cfc
I20260928 23:10:51.156916 5098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } }
I20260928 23:10:51.157079 5098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:51.157119 5098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49bcdd886dbb4b6ebe6b5a0ba63cf278, State: Initialized, Role: FOLLOWER
I20260928 23:10:51.157298 5098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } }
I20260928 23:10:51.157414 5098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:51.157461 5098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:51.157544 5098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:51.158092 5098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } }
I20260928 23:10:51.158206 5098 leader_election.cc:302] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49bcdd886dbb4b6ebe6b5a0ba63cf278; no voters:
I20260928 23:10:51.158393 5098 leader_election.cc:288] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:51.158414 5103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:51.158532 5103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [term 1 LEADER]: Becoming Leader. State: Replica: 49bcdd886dbb4b6ebe6b5a0ba63cf278, State: Running, Role: LEADER
I20260928 23:10:51.158668 5098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:51.158670 5103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } }
I20260928 23:10:51.159133 5104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } } }
I20260928 23:10:51.159230 5104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:51.159154 5105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49bcdd886dbb4b6ebe6b5a0ba63cf278. Latest consensus state: current_term: 1 leader_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bcdd886dbb4b6ebe6b5a0ba63cf278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44833 } } }
I20260928 23:10:51.159410 5105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:51.159427 5110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:51.159955 5110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:51.161650 5110 catalog_manager.cc:1347] Generated new cluster ID: d8ca7bd8027e4aa2992c0f69aa7149b4
I20260928 23:10:51.161701 5110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:51.162158 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:51.156297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:50363 (local address 127.0.0.1:44833)
0928 23:10:51.156493 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:51.156501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:51.156531 (+ 30us) server_negotiation.cc:407] Connection header received
0928 23:10:51.156899 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:51.156904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:51.156950 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:51.157064 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:51.157339 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:51.158714 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:51.159884 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:51.160504 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:51.160608 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:51.161393 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:51.161397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:51.161397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:51.161611 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:51.161651 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:51.161665 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:51.161793 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:51.161877 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 23:10:51.161994 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260928 23:10:51.163021 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:51.171236 5110 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:51.171478 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:51.164398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48777 (local address 127.0.0.1:44833)
0928 23:10:51.164454 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:51.164456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:51.168624 (+ 4168us) server_negotiation.cc:407] Connection header received
0928 23:10:51.168711 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:51.168711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:51.168727 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:51.168755 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:51.168927 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:51.169376 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:51.170175 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:51.170401 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:51.170486 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:51.171136 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:51.171137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:51.171137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:51.171154 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:51.171169 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:51.171174 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:51.171234 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:51.171359 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 23:10:51.171370 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:10:51.171944 5110 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:51.175927 5110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49bcdd886dbb4b6ebe6b5a0ba63cf278: Generated new TSK 0
I20260928 23:10:51.176092 5110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:51.178043 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5030
W20260928 23:10:51.182670 5015 connection.cc:535] client connection to 127.0.0.1:44833 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44833): Connection reset by peer (error 104)
2026-09-28T23:10:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:51 dist-test-slave-x2t8 krb5kdc[5129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:51 dist-test-slave-x2t8 krb5kdc[5129](info): set up 2 sockets
Sep 28 23:10:51 dist-test-slave-x2t8 krb5kdc[5129](info): commencing operation
krb5kdc: starting...
W20260928 23:10:52.212498 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:52 dist-test-slave-x2t8 krb5kdc[5129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 23:10:52.323614 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:52.401810 5142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:52.401997 5142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:52.402020 5142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:52.403147 5142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:52.403190 5142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:52.403204 5142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:52.403242 5142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:52.403270 5142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:52.404762 5142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:52.405002 5142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:52.405218 5142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:10:52.410073 5142 server_base.cc:1034] running on GCE node
W20260928 23:10:52.409965 5148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:52.409977 5147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:52.409966 5150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:52.410567 5142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:52.410851 5142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:52.412020 5142 hybrid_clock.cc:648] HybridClock initialized: now 1790637052412008 us; error 43 us; skew 500 ppm
Sep 28 23:10:52 dist-test-slave-x2t8 krb5kdc[5129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637052, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:52.414390 5142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 23:10:52.414839 5142 webserver.cc:460] Webserver started at http://127.0.0.1:44141/ using document root <none> and password file <none>
I20260928 23:10:52.415069 5142 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:52.415136 5142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:52.415284 5142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:52.416268 5142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "197600f4fe0f4985968344ab46d579b1"
format_stamp: "Formatted at 2026-09-28 23:10:52 on dist-test-slave-x2t8"
server_key: "66eade904e04518ccc282ae5f7263d30"
server_key_iv: "14cc3002e64a1b91e6aa79da105c3684"
server_key_version: "encryptionkey@0"
I20260928 23:10:52.416647 5142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "197600f4fe0f4985968344ab46d579b1"
format_stamp: "Formatted at 2026-09-28 23:10:52 on dist-test-slave-x2t8"
server_key: "66eade904e04518ccc282ae5f7263d30"
server_key_iv: "14cc3002e64a1b91e6aa79da105c3684"
server_key_version: "encryptionkey@0"
I20260928 23:10:52.417977 5142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 23:10:52.418643 5157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:52.418800 5142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:52.418905 5142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "197600f4fe0f4985968344ab46d579b1"
format_stamp: "Formatted at 2026-09-28 23:10:52 on dist-test-slave-x2t8"
server_key: "66eade904e04518ccc282ae5f7263d30"
server_key_iv: "14cc3002e64a1b91e6aa79da105c3684"
server_key_version: "encryptionkey@0"
I20260928 23:10:52.418975 5142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:52.422189 5142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:52.423871 5142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:52.424023 5142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:52.427913 5142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45477
I20260928 23:10:52.427942 5209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45477 every 8 connection(s)
I20260928 23:10:52.428246 5142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:52.429060 5210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:52.431061 5210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1: Bootstrap starting.
I20260928 23:10:52.431545 5210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:52.431763 5210 log.cc:824] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:52.432325 5210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1: No bootstrap required, opened a new log
I20260928 23:10:52.433856 5210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } }
I20260928 23:10:52.433990 5210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:52.434011 5210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197600f4fe0f4985968344ab46d579b1, State: Initialized, Role: FOLLOWER
I20260928 23:10:52.434104 5210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } }
I20260928 23:10:52.434154 5210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:52.434173 5210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:52.434197 5210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:52.434638 5210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } }
I20260928 23:10:52.434716 5210 leader_election.cc:302] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 197600f4fe0f4985968344ab46d579b1; no voters:
I20260928 23:10:52.434860 5210 leader_election.cc:288] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:52.434885 5213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:52.435000 5213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [term 1 LEADER]: Becoming Leader. State: Replica: 197600f4fe0f4985968344ab46d579b1, State: Running, Role: LEADER
I20260928 23:10:52.435114 5210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:52.435137 5213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } }
I20260928 23:10:52.435556 5215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "197600f4fe0f4985968344ab46d579b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } } }
I20260928 23:10:52.435655 5215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:52.435765 5214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 197600f4fe0f4985968344ab46d579b1. Latest consensus state: current_term: 1 leader_uuid: "197600f4fe0f4985968344ab46d579b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197600f4fe0f4985968344ab46d579b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45477 } } }
I20260928 23:10:52.435894 5214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:52.436232 5222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:52.436787 5222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:52.437696 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5142
I20260928 23:10:52.437819 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:52.438045 16209 external_mini_cluster.cc:1428] Setting key 4cc0f4ba642e7ba6e60200cfdd0c171a
I20260928 23:10:52.438297 5222 catalog_manager.cc:1347] Generated new cluster ID: dadd1b11f40c4d55ae2bd304e8da6be4
I20260928 23:10:52.438338 5222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:52.440097 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:52.438444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:34175 (local address 127.0.0.1:45477)
0928 23:10:52.438674 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:52.438682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:52.438711 (+ 29us) server_negotiation.cc:407] Connection header received
0928 23:10:52.438821 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:52.438826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:52.438853 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:52.438977 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:52.439605 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:52.439608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:52.439608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:52.439612 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:52.439650 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:52.439664 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:52.439719 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:52.439816 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 23:10:52.439939 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260928 23:10:52.440913 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:52.443365 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:52.442293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:47955 (local address 127.0.0.1:45477)
0928 23:10:52.442329 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:52.442331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:52.442451 (+ 120us) server_negotiation.cc:407] Connection header received
0928 23:10:52.442505 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:52.442506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:52.442522 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:52.442587 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:52.443153 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:52.443155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:52.443155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:52.443159 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:52.443172 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:52.443177 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:52.443238 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:52.443292 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 23:10:52.443305 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 23:10:52.450701 5222 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:52.450843 5222 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:52.454550 5222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 197600f4fe0f4985968344ab46d579b1: Generated new TSK 0
I20260928 23:10:52.454749 5222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:52.461467 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5142
2026-09-28T23:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:52 dist-test-slave-x2t8 krb5kdc[5241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:52 dist-test-slave-x2t8 krb5kdc[5241](info): set up 2 sockets
Sep 28 23:10:52 dist-test-slave-x2t8 krb5kdc[5241](info): commencing operation
krb5kdc: starting...
W20260928 23:10:53.484337 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:53.595402 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46883
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:53.674082 5254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:53.674307 5254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:53.674345 5254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:53.675483 5254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:53.675549 5254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:53.675575 5254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:53.675606 5254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:53.675628 5254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:53.677143 5254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:53.677440 5254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:53.677660 5254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:53.682366 5260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:53.682432 5262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:53.682461 5259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:53.682703 5254 server_base.cc:1034] running on GCE node
I20260928 23:10:53.682933 5254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:53.683274 5254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:53.684419 5254 hybrid_clock.cc:648] HybridClock initialized: now 1790637053684400 us; error 39 us; skew 500 ppm
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637053, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:53.686971 5254 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:53.687439 5254 webserver.cc:460] Webserver started at http://127.15.212.126:34653/ using document root <none> and password file <none>
I20260928 23:10:53.687682 5254 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:53.687726 5254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:53.687875 5254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:53.688963 5254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "b3c7b926dab048a9a03322b107e8f2e6"
format_stamp: "Formatted at 2026-09-28 23:10:53 on dist-test-slave-x2t8"
server_key: "09730e2e0c34cd9a2f653ee9d0e37d75"
server_key_iv: "c24d3141306bfd98f4d4d8e52e948b41"
server_key_version: "encryptionkey@0"
I20260928 23:10:53.689325 5254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "b3c7b926dab048a9a03322b107e8f2e6"
format_stamp: "Formatted at 2026-09-28 23:10:53 on dist-test-slave-x2t8"
server_key: "09730e2e0c34cd9a2f653ee9d0e37d75"
server_key_iv: "c24d3141306bfd98f4d4d8e52e948b41"
server_key_version: "encryptionkey@0"
I20260928 23:10:53.690603 5254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:10:53.691224 5269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:53.691406 5254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:10:53.691493 5254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "b3c7b926dab048a9a03322b107e8f2e6"
format_stamp: "Formatted at 2026-09-28 23:10:53 on dist-test-slave-x2t8"
server_key: "09730e2e0c34cd9a2f653ee9d0e37d75"
server_key_iv: "c24d3141306bfd98f4d4d8e52e948b41"
server_key_version: "encryptionkey@0"
I20260928 23:10:53.691560 5254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:53.703197 5254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:53.705417 5254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:53.705571 5254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:53.709425 5254 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46883
I20260928 23:10:53.709455 5321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46883 every 8 connection(s)
I20260928 23:10:53.709824 5254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:53.710632 5322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:53.712872 5322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6: Bootstrap starting.
I20260928 23:10:53.713425 5322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:53.713709 5322 log.cc:824] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:53.714380 5322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6: No bootstrap required, opened a new log
I20260928 23:10:53.715911 5322 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } }
I20260928 23:10:53.716040 5322 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:53.716063 5322 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3c7b926dab048a9a03322b107e8f2e6, State: Initialized, Role: FOLLOWER
I20260928 23:10:53.716233 5322 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } }
I20260928 23:10:53.716291 5322 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:53.716328 5322 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:53.716377 5322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:53.717192 5322 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } }
I20260928 23:10:53.717298 5322 leader_election.cc:302] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3c7b926dab048a9a03322b107e8f2e6; no voters:
I20260928 23:10:53.717509 5322 leader_election.cc:288] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:53.717525 5325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:53.717720 5325 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [term 1 LEADER]: Becoming Leader. State: Replica: b3c7b926dab048a9a03322b107e8f2e6, State: Running, Role: LEADER
I20260928 23:10:53.717778 5322 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:53.717857 5325 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } }
I20260928 23:10:53.718282 5326 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3c7b926dab048a9a03322b107e8f2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } } }
I20260928 23:10:53.718288 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3c7b926dab048a9a03322b107e8f2e6. Latest consensus state: current_term: 1 leader_uuid: "b3c7b926dab048a9a03322b107e8f2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3c7b926dab048a9a03322b107e8f2e6" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46883 } } }
I20260928 23:10:53.718387 5326 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:53.718394 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:53.718619 5334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:53.719154 5334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:53.719913 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5254
I20260928 23:10:53.720039 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:53.720237 16209 external_mini_cluster.cc:1428] Setting key 23592404261ee7b0054f14c3fac9575f
I20260928 23:10:53.720935 5334 catalog_manager.cc:1347] Generated new cluster ID: af5ecce5abe548b786a623b66af65901
I20260928 23:10:53.720984 5334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:53.725915 5334 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:53.726596 5334 catalog_manager.cc:1504] Loading token signing keys...
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:53.731632 5334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3c7b926dab048a9a03322b107e8f2e6: Generated new TSK 0
I20260928 23:10:53.731778 5334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:53.735222 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:53.720723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36557 (local address 127.15.212.126:46883)
0928 23:10:53.720873 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:53.720882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:53.725830 (+ 4948us) server_negotiation.cc:407] Connection header received
0928 23:10:53.725943 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:53.725949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:53.726020 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:53.726171 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:53.726858 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:53.728019 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:53.728898 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:53.729289 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:53.729412 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:53.731872 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:53.731876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:53.731876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:53.731886 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:53.733607 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:53.734013 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:53.734016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:53.734017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:53.734106 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:53.734372 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:53.734372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:53.734373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:53.734606 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:53.734792 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:53.734930 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 23:10:53.735040 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260928 23:10:53.736308 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:53.743616 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:53.737661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34203 (local address 127.15.212.126:46883)
0928 23:10:53.737720 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:53.737722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:53.737843 (+ 121us) server_negotiation.cc:407] Connection header received
0928 23:10:53.737916 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:53.737916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:53.737926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:53.737973 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:53.738464 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:53.738964 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:53.739691 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:53.739926 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:53.740019 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:53.741176 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:53.741177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:53.741178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:53.741185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:53.742461 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:53.742831 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:53.742832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:53.742832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:53.742888 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:53.743113 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:53.743114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:53.743114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:53.743222 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:53.743348 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:53.743453 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 23:10:53.743513 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 23:10:53.745672 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5254
W20260928 23:10:53.749934 5239 connection.cc:535] client connection to 127.15.212.126:46883 recv error: Network error: failed to read from TLS socket (remote: 127.15.212.126:46883): Connection reset by peer (error 104)
2026-09-28T23:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5353](info): set up 2 sockets
Sep 28 23:10:53 dist-test-slave-x2t8 krb5kdc[5353](info): commencing operation
krb5kdc: starting...
W20260928 23:10:54.776636 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:54 dist-test-slave-x2t8 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:54.887387 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39635
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:39635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:54.965852 5366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:54.966075 5366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:54.966111 5366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:54.967255 5366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:54.967322 5366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:54.967347 5366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:54.967368 5366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:54.967396 5366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:54.968885 5366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:39635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:39635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:54.969159 5366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:54.969383 5366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:54.974150 5372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:54.974226 5366 server_base.cc:1034] running on GCE node
W20260928 23:10:54.974161 5371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:54.974141 5374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:54.974638 5366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:54.974956 5366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:54.976125 5366 hybrid_clock.cc:648] HybridClock initialized: now 1790637054976111 us; error 42 us; skew 500 ppm
Sep 28 23:10:54 dist-test-slave-x2t8 krb5kdc[5353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637054, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:54.981900 5366 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:54.982342 5366 webserver.cc:460] Webserver started at http://127.15.212.126:41857/ using document root <none> and password file <none>
I20260928 23:10:54.982568 5366 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:54.982635 5366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:54.982784 5366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:54.983777 5366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "63ef1de6966849b4819b0fd78de77410"
format_stamp: "Formatted at 2026-09-28 23:10:54 on dist-test-slave-x2t8"
server_key: "c871bb28b6f3fa1e9009444ff00c6244"
server_key_iv: "e768900c8f924f65f68fee853ef3cf08"
server_key_version: "encryptionkey@0"
I20260928 23:10:54.984133 5366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "63ef1de6966849b4819b0fd78de77410"
format_stamp: "Formatted at 2026-09-28 23:10:54 on dist-test-slave-x2t8"
server_key: "c871bb28b6f3fa1e9009444ff00c6244"
server_key_iv: "e768900c8f924f65f68fee853ef3cf08"
server_key_version: "encryptionkey@0"
I20260928 23:10:54.985479 5366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:54.986176 5381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:54.986338 5366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:54.986440 5366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "63ef1de6966849b4819b0fd78de77410"
format_stamp: "Formatted at 2026-09-28 23:10:54 on dist-test-slave-x2t8"
server_key: "c871bb28b6f3fa1e9009444ff00c6244"
server_key_iv: "e768900c8f924f65f68fee853ef3cf08"
server_key_version: "encryptionkey@0"
I20260928 23:10:54.986509 5366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:54.995893 5366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:54.997625 5366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:54.997776 5366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:55.001736 5366 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:39635
I20260928 23:10:55.001782 5433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:39635 every 8 connection(s)
I20260928 23:10:55.002125 5366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:55.002909 5434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:55.005069 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410: Bootstrap starting.
I20260928 23:10:55.005596 5434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:55.005893 5434 log.cc:824] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:55.006558 5434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410: No bootstrap required, opened a new log
I20260928 23:10:55.008056 5434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } }
I20260928 23:10:55.008175 5434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:55.008195 5434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ef1de6966849b4819b0fd78de77410, State: Initialized, Role: FOLLOWER
I20260928 23:10:55.008368 5434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } }
I20260928 23:10:55.008427 5434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:55.008510 5434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:55.008560 5434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:55.009332 5434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } }
I20260928 23:10:55.009466 5434 leader_election.cc:302] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63ef1de6966849b4819b0fd78de77410; no voters:
I20260928 23:10:55.009632 5434 leader_election.cc:288] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:55.009667 5437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:55.009858 5437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [term 1 LEADER]: Becoming Leader. State: Replica: 63ef1de6966849b4819b0fd78de77410, State: Running, Role: LEADER
I20260928 23:10:55.009863 5434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:55.009958 5437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } }
I20260928 23:10:55.010404 5438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63ef1de6966849b4819b0fd78de77410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } } }
I20260928 23:10:55.010418 5439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63ef1de6966849b4819b0fd78de77410. Latest consensus state: current_term: 1 leader_uuid: "63ef1de6966849b4819b0fd78de77410" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ef1de6966849b4819b0fd78de77410" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 39635 } } }
I20260928 23:10:55.010506 5438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:55.010519 5439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410 [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:55.010757 5446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:55.011327 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5366
I20260928 23:10:55.011468 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:55.011469 5446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:55.011675 16209 external_mini_cluster.cc:1428] Setting key e25b91029cd9d034ba236e65da26486e
I20260928 23:10:55.013248 5446 catalog_manager.cc:1347] Generated new cluster ID: 7bd55037d853438b896da4a19a6a671b
I20260928 23:10:55.013302 5446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:55.016610 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:55.012274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53625 (local address 127.15.212.126:39635)
0928 23:10:55.012503 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:55.012511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:55.012546 (+ 35us) server_negotiation.cc:407] Connection header received
0928 23:10:55.012648 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:55.012654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:55.012716 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:55.012843 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:55.013059 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:55.014021 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:55.014782 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:55.015181 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:55.015304 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:55.015977 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:55.015981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:55.015981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:55.016148 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:55.016175 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:55.016185 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:55.016274 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:55.016356 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 23:10:55.016383 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260928 23:10:55.017587 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:55.018210 5446 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:55.018846 5446 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:55.022126 5446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63ef1de6966849b4819b0fd78de77410: Generated new TSK 0
I20260928 23:10:55.022277 5446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:55.024973 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:55.020778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60657 (local address 127.15.212.126:39635)
0928 23:10:55.020820 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:55.020822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:55.021790 (+ 968us) server_negotiation.cc:407] Connection header received
0928 23:10:55.021892 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:55.021893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:55.021911 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:55.021960 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:55.022109 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:55.022697 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:55.023772 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:55.023969 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:55.024019 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:55.024706 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:55.024707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:55.024707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:55.024710 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:55.024722 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:55.024726 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:55.024770 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:55.024838 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 23:10:55.024878 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:10:55.026713 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5366
W20260928 23:10:55.031159 5351 connection.cc:535] client connection to 127.15.212.126:39635 recv error: Network error: failed to read from TLS socket (remote: 127.15.212.126:39635): Connection reset by peer (error 104)
2026-09-28T23:10:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:55 dist-test-slave-x2t8 krb5kdc[5465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:55 dist-test-slave-x2t8 krb5kdc[5465](info): set up 2 sockets
Sep 28 23:10:55 dist-test-slave-x2t8 krb5kdc[5465](info): commencing operation
krb5kdc: starting...
W20260928 23:10:56.056613 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:56 dist-test-slave-x2t8 krb5kdc[5465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:56.171262 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45883
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:34875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:45883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:56.249367 5478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:56.249553 5478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:56.249574 5478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:56.250716 5478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:56.250756 5478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:56.250770 5478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:56.250783 5478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:56.250836 5478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:56.252238 5478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:34875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:45883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:56.252506 5478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:56.252756 5478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:56.257368 5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:56.257369 5483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:56.257406 5478 server_base.cc:1034] running on GCE node
W20260928 23:10:56.257360 5486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:56.257862 5478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:56.258136 5478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:56.259272 5478 hybrid_clock.cc:648] HybridClock initialized: now 1790637056259260 us; error 28 us; skew 500 ppm
Sep 28 23:10:56 dist-test-slave-x2t8 krb5kdc[5465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637056, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:56.261637 5478 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:56.262038 5478 webserver.cc:460] Webserver started at http://127.15.212.126:32863/ using document root <none> and password file <none>
I20260928 23:10:56.262270 5478 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:56.262313 5478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:56.262466 5478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:56.263499 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "65bdefd2e96244afa7c509e54ae2a03c"
format_stamp: "Formatted at 2026-09-28 23:10:56 on dist-test-slave-x2t8"
server_key: "44833f7a8c2d25f70aa1f489a8481687"
server_key_iv: "a401ad3fdb35c1fe2b4ff76efbb3aa06"
server_key_version: "encryptionkey@0"
I20260928 23:10:56.263824 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "65bdefd2e96244afa7c509e54ae2a03c"
format_stamp: "Formatted at 2026-09-28 23:10:56 on dist-test-slave-x2t8"
server_key: "44833f7a8c2d25f70aa1f489a8481687"
server_key_iv: "a401ad3fdb35c1fe2b4ff76efbb3aa06"
server_key_version: "encryptionkey@0"
I20260928 23:10:56.265129 5478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:56.265820 5493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:56.266004 5478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:56.266090 5478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "65bdefd2e96244afa7c509e54ae2a03c"
format_stamp: "Formatted at 2026-09-28 23:10:56 on dist-test-slave-x2t8"
server_key: "44833f7a8c2d25f70aa1f489a8481687"
server_key_iv: "a401ad3fdb35c1fe2b4ff76efbb3aa06"
server_key_version: "encryptionkey@0"
I20260928 23:10:56.266160 5478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:56.269322 5478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:56.271020 5478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:56.271169 5478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:56.274940 5478 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:45883
I20260928 23:10:56.275017 5545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:45883 every 8 connection(s)
I20260928 23:10:56.275352 5478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:56.276285 5546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:56.278416 5546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c: Bootstrap starting.
I20260928 23:10:56.278966 5546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:56.279238 5546 log.cc:824] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:56.279865 5546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c: No bootstrap required, opened a new log
I20260928 23:10:56.281454 5546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } }
I20260928 23:10:56.281594 5546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:56.281639 5546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65bdefd2e96244afa7c509e54ae2a03c, State: Initialized, Role: FOLLOWER
I20260928 23:10:56.281818 5546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } }
I20260928 23:10:56.281906 5546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:56.281946 5546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:56.281991 5546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:56.282778 5546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } }
I20260928 23:10:56.282887 5546 leader_election.cc:302] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65bdefd2e96244afa7c509e54ae2a03c; no voters:
I20260928 23:10:56.283063 5546 leader_election.cc:288] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:56.283116 5549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:56.283262 5549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [term 1 LEADER]: Becoming Leader. State: Replica: 65bdefd2e96244afa7c509e54ae2a03c, State: Running, Role: LEADER
I20260928 23:10:56.283301 5546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:56.283370 5549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } }
I20260928 23:10:56.283712 5551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65bdefd2e96244afa7c509e54ae2a03c. Latest consensus state: current_term: 1 leader_uuid: "65bdefd2e96244afa7c509e54ae2a03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } } }
I20260928 23:10:56.283696 5550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65bdefd2e96244afa7c509e54ae2a03c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65bdefd2e96244afa7c509e54ae2a03c" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 45883 } } }
I20260928 23:10:56.283818 5551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:56.283891 5550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:56.284277 5557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:56.284783 5557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:56.285382 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5478
I20260928 23:10:56.285493 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:56.285703 16209 external_mini_cluster.cc:1428] Setting key 6ea91550a6070fdd208bdea382623cad
I20260928 23:10:56.286347 5557 catalog_manager.cc:1347] Generated new cluster ID: 211e33e2425b45e18f7157ae7fa043c3
I20260928 23:10:56.286396 5557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:56.287846 5569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:56.286168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37147 (local address 127.15.212.126:45883)
0928 23:10:56.286316 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:56.286325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:56.286552 (+ 227us) server_negotiation.cc:407] Connection header received
0928 23:10:56.286664 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:56.286671 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:56.286698 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:56.286825 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:56.287440 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:56.287443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:56.287444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:56.287447 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:56.287486 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:56.287499 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:56.287564 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:56.287654 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 23:10:56.287690 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260928 23:10:56.292892 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:56.295297 5569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:56.294263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38651 (local address 127.15.212.126:45883)
0928 23:10:56.294291 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:56.294293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:56.294419 (+ 126us) server_negotiation.cc:407] Connection header received
0928 23:10:56.294480 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:56.294481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:56.294486 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:56.294546 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:56.295095 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:56.295096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:56.295097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:56.295100 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:56.295114 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:56.295118 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:56.295170 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:56.295226 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 23:10:56.295237 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260928 23:10:56.300119 5557 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:56.300263 5557 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:56.310276 5557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65bdefd2e96244afa7c509e54ae2a03c: Generated new TSK 0
I20260928 23:10:56.310448 5557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:56.320180 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5478
2026-09-28T23:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:56 dist-test-slave-x2t8 krb5kdc[5577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:56 dist-test-slave-x2t8 krb5kdc[5577](info): set up 2 sockets
Sep 28 23:10:56 dist-test-slave-x2t8 krb5kdc[5577](info): commencing operation
krb5kdc: starting...
W20260928 23:10:57.356541 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:57.470649 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46173
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:46173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:57.548362 5590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:57.548604 5590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:57.548640 5590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:57.549769 5590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:57.549836 5590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:57.549860 5590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:57.549887 5590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:57.549916 5590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:57.551329 5590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:46173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:46173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:57.551585 5590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:57.551798 5590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:57.556322 5598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:57.556372 5595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:57.556322 5596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:57.556566 5590 server_base.cc:1034] running on GCE node
I20260928 23:10:57.556767 5590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:57.557036 5590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:57.558172 5590 hybrid_clock.cc:648] HybridClock initialized: now 1790637057558162 us; error 27 us; skew 500 ppm
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637057, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:57.560580 5590 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:57.561019 5590 webserver.cc:460] Webserver started at http://127.15.212.126:43619/ using document root <none> and password file <none>
I20260928 23:10:57.561257 5590 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:57.561299 5590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:57.561457 5590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:57.562486 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "441ee0b991084dfda577be5f6a61b40f"
format_stamp: "Formatted at 2026-09-28 23:10:57 on dist-test-slave-x2t8"
server_key: "b693027c15811a31cbce10d849403acd"
server_key_iv: "f85ff01c8d44dc4350b59a1b827ff4cc"
server_key_version: "encryptionkey@0"
I20260928 23:10:57.562834 5590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "441ee0b991084dfda577be5f6a61b40f"
format_stamp: "Formatted at 2026-09-28 23:10:57 on dist-test-slave-x2t8"
server_key: "b693027c15811a31cbce10d849403acd"
server_key_iv: "f85ff01c8d44dc4350b59a1b827ff4cc"
server_key_version: "encryptionkey@0"
I20260928 23:10:57.564075 5590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:10:57.564889 5605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:57.565069 5590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 23:10:57.565132 5590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "441ee0b991084dfda577be5f6a61b40f"
format_stamp: "Formatted at 2026-09-28 23:10:57 on dist-test-slave-x2t8"
server_key: "b693027c15811a31cbce10d849403acd"
server_key_iv: "f85ff01c8d44dc4350b59a1b827ff4cc"
server_key_version: "encryptionkey@0"
I20260928 23:10:57.565222 5590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:57.588377 5590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:57.590212 5590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:57.590368 5590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:57.594292 5590 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:46173
I20260928 23:10:57.594352 5657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:46173 every 8 connection(s)
I20260928 23:10:57.594679 5590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:57.595181 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5590
I20260928 23:10:57.595299 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:57.595504 16209 external_mini_cluster.cc:1428] Setting key 9cb928563fab301be1e43af2636a10e7
I20260928 23:10:57.595626 5658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:57.598873 5658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f: Bootstrap starting.
I20260928 23:10:57.599545 5658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:57.599850 5658 log.cc:824] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:57.600677 5658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f: No bootstrap required, opened a new log
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:10:57.603145 5658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } }
I20260928 23:10:57.603331 5658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:57.603379 5658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 441ee0b991084dfda577be5f6a61b40f, State: Initialized, Role: FOLLOWER
I20260928 23:10:57.603528 5658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } }
I20260928 23:10:57.603646 5658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:57.603722 5658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:57.603797 5658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:57.604317 5658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } }
I20260928 23:10:57.604435 5658 leader_election.cc:302] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 441ee0b991084dfda577be5f6a61b40f; no voters:
I20260928 23:10:57.604648 5658 leader_election.cc:288] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:57.604676 5663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:57.604791 5663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [term 1 LEADER]: Becoming Leader. State: Replica: 441ee0b991084dfda577be5f6a61b40f, State: Running, Role: LEADER
I20260928 23:10:57.604914 5658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:57.604921 5663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } }
I20260928 23:10:57.605332 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 441ee0b991084dfda577be5f6a61b40f. Latest consensus state: current_term: 1 leader_uuid: "441ee0b991084dfda577be5f6a61b40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } } }
I20260928 23:10:57.605424 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:57.605327 5664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "441ee0b991084dfda577be5f6a61b40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441ee0b991084dfda577be5f6a61b40f" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 46173 } } }
I20260928 23:10:57.605710 5664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:57.606056 5669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:57.606585 5669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:57.606626 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:57.596108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42439 (local address 127.15.212.126:46173)
0928 23:10:57.596248 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:57.596255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:57.596277 (+ 22us) server_negotiation.cc:407] Connection header received
0928 23:10:57.596813 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:57.596819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:57.596864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:57.596969 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:57.597775 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:57.598619 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:57.599678 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:57.600145 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:57.600257 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:57.602788 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:57.602794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:57.602794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:57.602810 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:57.604544 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:57.604938 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:57.604941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:57.604941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:57.605089 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:57.605268 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:57.605269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:57.605269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:57.605595 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:57.605752 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:57.606279 (+ 527us) server_negotiation.cc:299] Negotiation successful
0928 23:10:57.606420 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260928 23:10:57.607509 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:57.608145 5669 catalog_manager.cc:1347] Generated new cluster ID: a5b11c1bdef747a7b020c2f4bc8fdb86
I20260928 23:10:57.608183 5669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:57.616014 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:57.608975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46441 (local address 127.15.212.126:46173)
0928 23:10:57.609004 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:57.609005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:57.609165 (+ 160us) server_negotiation.cc:407] Connection header received
0928 23:10:57.609222 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:57.609223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:57.609232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:57.609288 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:57.609876 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:57.610579 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:57.611582 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:57.611879 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:57.611934 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:57.613330 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:57.613331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:57.613332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:10:57.613338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:57.614865 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:57.615265 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:57.615266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:57.615266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:57.615316 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:10:57.615526 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:10:57.615527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:10:57.615527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:10:57.615634 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:57.615715 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:57.615860 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 23:10:57.615926 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260928 23:10:57.634697 5669 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:57.635264 5669 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:57.637871 5669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 441ee0b991084dfda577be5f6a61b40f: Generated new TSK 0
I20260928 23:10:57.638060 5669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:57.645539 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5590
2026-09-28T23:10:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5689](info): set up 2 sockets
Sep 28 23:10:57 dist-test-slave-x2t8 krb5kdc[5689](info): commencing operation
krb5kdc: starting...
W20260928 23:10:58.672578 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:10:58 dist-test-slave-x2t8 krb5kdc[5689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:10:58.783449 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44199
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:44199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:10:58.861475 5702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:10:58.861649 5702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:10:58.861670 5702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:10:58.862746 5702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:10:58.862785 5702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:10:58.862799 5702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:10:58.862811 5702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:10:58.862851 5702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:10:58.864252 5702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:44199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:44199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:10:58.864494 5702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:10:58.864704 5702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:10:58.869508 5708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:10:58.869570 5710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:58.869606 5702 server_base.cc:1034] running on GCE node
W20260928 23:10:58.869493 5707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:10:58.869907 5702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:10:58.870194 5702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:10:58.871383 5702 hybrid_clock.cc:648] HybridClock initialized: now 1790637058871364 us; error 36 us; skew 500 ppm
Sep 28 23:10:58 dist-test-slave-x2t8 krb5kdc[5689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637058, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:10:58.873720 5702 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:10:58.874163 5702 webserver.cc:460] Webserver started at http://127.15.212.126:42615/ using document root <none> and password file <none>
I20260928 23:10:58.874377 5702 fs_manager.cc:359] Metadata directory not provided
I20260928 23:10:58.874444 5702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:10:58.874603 5702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:10:58.875605 5702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "77d09cbeaf4146dba46a40490fe687fc"
format_stamp: "Formatted at 2026-09-28 23:10:58 on dist-test-slave-x2t8"
server_key: "c6f20ffa4b4b4192a55f92f8600ad9b4"
server_key_iv: "38f33decedd4c88da0a8e6c4825eb066"
server_key_version: "encryptionkey@0"
I20260928 23:10:58.875958 5702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "77d09cbeaf4146dba46a40490fe687fc"
format_stamp: "Formatted at 2026-09-28 23:10:58 on dist-test-slave-x2t8"
server_key: "c6f20ffa4b4b4192a55f92f8600ad9b4"
server_key_iv: "38f33decedd4c88da0a8e6c4825eb066"
server_key_version: "encryptionkey@0"
I20260928 23:10:58.877434 5702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:10:58.878125 5717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:10:58.878304 5702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:10:58.878399 5702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "77d09cbeaf4146dba46a40490fe687fc"
format_stamp: "Formatted at 2026-09-28 23:10:58 on dist-test-slave-x2t8"
server_key: "c6f20ffa4b4b4192a55f92f8600ad9b4"
server_key_iv: "38f33decedd4c88da0a8e6c4825eb066"
server_key_version: "encryptionkey@0"
I20260928 23:10:58.878484 5702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:10:58.892545 5702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:10:58.894232 5702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:10:58.894390 5702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:10:58.898320 5702 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:44199
I20260928 23:10:58.898358 5769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:44199 every 8 connection(s)
I20260928 23:10:58.898699 5702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:10:58.899411 5770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:10:58.901592 5770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc: Bootstrap starting.
I20260928 23:10:58.902102 5770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc: Neither blocks nor log segments found. Creating new log.
I20260928 23:10:58.902330 5770 log.cc:824] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc: Log is configured to *not* fsync() on all Append() calls
I20260928 23:10:58.903054 5770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc: No bootstrap required, opened a new log
I20260928 23:10:58.904521 5770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } }
I20260928 23:10:58.904644 5770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:10:58.904664 5770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d09cbeaf4146dba46a40490fe687fc, State: Initialized, Role: FOLLOWER
I20260928 23:10:58.904757 5770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } }
I20260928 23:10:58.904803 5770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:10:58.904824 5770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:10:58.904853 5770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:10:58.905577 5770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } }
I20260928 23:10:58.905658 5770 leader_election.cc:302] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77d09cbeaf4146dba46a40490fe687fc; no voters:
I20260928 23:10:58.905767 5770 leader_election.cc:288] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:10:58.905829 5773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:10:58.906002 5773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [term 1 LEADER]: Becoming Leader. State: Replica: 77d09cbeaf4146dba46a40490fe687fc, State: Running, Role: LEADER
I20260928 23:10:58.906049 5770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:10:58.906126 5773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } }
I20260928 23:10:58.906497 5774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77d09cbeaf4146dba46a40490fe687fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } } }
I20260928 23:10:58.906498 5775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77d09cbeaf4146dba46a40490fe687fc. Latest consensus state: current_term: 1 leader_uuid: "77d09cbeaf4146dba46a40490fe687fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d09cbeaf4146dba46a40490fe687fc" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 44199 } } }
I20260928 23:10:58.906677 5774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:58.906685 5775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc [sys.catalog]: This master's current role is: LEADER
I20260928 23:10:58.907167 5782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:10:58.907640 5782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:10:58.907691 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5702
I20260928 23:10:58.907809 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:10:58.907994 16209 external_mini_cluster.cc:1428] Setting key ecd825d061616bb88f75b8d24a20f39e
I20260928 23:10:58.909184 5782 catalog_manager.cc:1347] Generated new cluster ID: 4016d06a93454935927ef05bb6122ff8
I20260928 23:10:58.909233 5782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:10:58.914116 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:58.908664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46323 (local address 127.15.212.126:44199)
0928 23:10:58.908835 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:58.908843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:58.908897 (+ 54us) server_negotiation.cc:407] Connection header received
0928 23:10:58.909000 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:58.909006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:58.909058 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:58.909184 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:58.909486 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:58.910830 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:58.911966 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:58.912573 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:58.912665 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:58.913390 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:58.913393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:58.913393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:58.913588 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:58.913623 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:58.913635 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:58.913695 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:58.913783 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 23:10:58.913927 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":109,"threads_started":1}
I20260928 23:10:58.914876 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:10:58.919543 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:10:58.916176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50091 (local address 127.15.212.126:44199)
0928 23:10:58.916202 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0928 23:10:58.916203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:10:58.916433 (+ 230us) server_negotiation.cc:407] Connection header received
0928 23:10:58.916486 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:10:58.916486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:10:58.916494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:10:58.916551 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:10:58.916675 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:58.917330 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:58.918350 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:10:58.918658 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:10:58.918712 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:10:58.919318 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:10:58.919319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:10:58.919319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:10:58.919325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:10:58.919336 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:10:58.919340 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:10:58.919393 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 23:10:58.919449 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 23:10:58.919470 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260928 23:10:58.921128 5782 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:10:58.921627 5782 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:10:58.924117 5782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77d09cbeaf4146dba46a40490fe687fc: Generated new TSK 0
I20260928 23:10:58.924252 5782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:10:58.928807 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5702
2026-09-28T23:10:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:10:58 dist-test-slave-x2t8 krb5kdc[5801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:10:58 dist-test-slave-x2t8 krb5kdc[5801](info): set up 2 sockets
Sep 28 23:10:58 dist-test-slave-x2t8 krb5kdc[5801](info): commencing operation
krb5kdc: starting...
W20260928 23:10:59.956421 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:11:00 dist-test-slave-x2t8 krb5kdc[5801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:11:00.071593 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40453
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:11:00.154553 5814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:00.154768 5814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:00.154804 5814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:00.155961 5814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:00.156026 5814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:00.156050 5814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:00.156072 5814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:00.156100 5814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:00.157569 5814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:00.157840 5814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:00.158056 5814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:11:00.162638 5820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:00.162653 5819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:00.162650 5822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:00.162988 5814 server_base.cc:1034] running on GCE node
I20260928 23:11:00.163224 5814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:00.163499 5814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:00.164646 5814 hybrid_clock.cc:648] HybridClock initialized: now 1790637060164624 us; error 36 us; skew 500 ppm
Sep 28 23:11:00 dist-test-slave-x2t8 krb5kdc[5801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637060, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:00.172251 5814 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:11:00.172727 5814 webserver.cc:460] Webserver started at http://127.15.212.126:46549/ using document root <none> and password file <none>
I20260928 23:11:00.172962 5814 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:00.173009 5814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:00.173179 5814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:00.174202 5814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "7c77117a2a154ae2991896af67c57667"
format_stamp: "Formatted at 2026-09-28 23:11:00 on dist-test-slave-x2t8"
server_key: "d152be9a9df7ed5900e38892fd96563a"
server_key_iv: "2d1c590cd19b1ab80264d2bbedde1e8f"
server_key_version: "encryptionkey@0"
I20260928 23:11:00.174561 5814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "7c77117a2a154ae2991896af67c57667"
format_stamp: "Formatted at 2026-09-28 23:11:00 on dist-test-slave-x2t8"
server_key: "d152be9a9df7ed5900e38892fd96563a"
server_key_iv: "2d1c590cd19b1ab80264d2bbedde1e8f"
server_key_version: "encryptionkey@0"
I20260928 23:11:00.175866 5814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:11:00.176617 5829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:00.176800 5814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:11:00.176890 5814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "7c77117a2a154ae2991896af67c57667"
format_stamp: "Formatted at 2026-09-28 23:11:00 on dist-test-slave-x2t8"
server_key: "d152be9a9df7ed5900e38892fd96563a"
server_key_iv: "2d1c590cd19b1ab80264d2bbedde1e8f"
server_key_version: "encryptionkey@0"
I20260928 23:11:00.176962 5814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:00.180142 5814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:00.182379 5814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:00.182539 5814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:00.186579 5814 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:40453
I20260928 23:11:00.186633 5881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:40453 every 8 connection(s)
I20260928 23:11:00.186934 5814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:11:00.187798 5882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:00.189962 5882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667: Bootstrap starting.
I20260928 23:11:00.190503 5882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:00.190778 5882 log.cc:824] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:00.191383 5882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667: No bootstrap required, opened a new log
I20260928 23:11:00.192993 5882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } }
I20260928 23:11:00.193138 5882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 23:11:00.193162 5882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c77117a2a154ae2991896af67c57667, State: Initialized, Role: FOLLOWER
I20260928 23:11:00.193342 5882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } }
I20260928 23:11:00.193401 5882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 23:11:00.193452 5882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 23:11:00.193500 5882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:00.194296 5882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } }
I20260928 23:11:00.194399 5882 leader_election.cc:302] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c77117a2a154ae2991896af67c57667; no voters:
I20260928 23:11:00.194579 5882 leader_election.cc:288] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 23:11:00.194618 5885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:11:00.194778 5885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [term 1 LEADER]: Becoming Leader. State: Replica: 7c77117a2a154ae2991896af67c57667, State: Running, Role: LEADER
I20260928 23:11:00.194819 5882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:00.194941 5885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } }
I20260928 23:11:00.195327 5886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c77117a2a154ae2991896af67c57667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } } }
I20260928 23:11:00.195313 5887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c77117a2a154ae2991896af67c57667. Latest consensus state: current_term: 1 leader_uuid: "7c77117a2a154ae2991896af67c57667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c77117a2a154ae2991896af67c57667" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 40453 } } }
I20260928 23:11:00.195456 5887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:00.195442 5886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:00.195959 5895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:11:00.195991 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5814
I20260928 23:11:00.196122 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:11:00.196358 16209 external_mini_cluster.cc:1428] Setting key fb7894b0b7ddc7732ac9a2b8d7bc7c10
I20260928 23:11:00.196486 5895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:11:00.198263 5895 catalog_manager.cc:1347] Generated new cluster ID: 4ff7ab5d6d594653b1f61c5dcc6fba97
I20260928 23:11:00.198318 5895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:11:00.201031 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:00.197013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48297 (local address 127.15.212.126:40453)
0928 23:11:00.197406 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:00.197414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:00.197443 (+ 29us) server_negotiation.cc:407] Connection header received
0928 23:11:00.197557 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:00.197563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:00.197592 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:00.197736 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:00.200630 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:00.200632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:00.200633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:11:00.200635 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:00.200659 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:11:00.200668 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:00.200741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:00.200815 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 23:11:00.200856 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":73,"threads_started":1}
I20260928 23:11:00.201906 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:11:00.205593 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:00.203171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36267 (local address 127.15.212.126:40453)
0928 23:11:00.203204 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:00.203205 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:00.204600 (+ 1395us) server_negotiation.cc:407] Connection header received
0928 23:11:00.204680 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:00.204681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:00.204686 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:00.204740 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:00.205333 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:00.205334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:00.205334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 23:11:00.205339 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:00.205352 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 23:11:00.205356 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:00.205410 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:00.205466 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 23:11:00.205480 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:11:00.206799 5895 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:11:00.206941 5895 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:11:00.213356 5895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c77117a2a154ae2991896af67c57667: Generated new TSK 0
I20260928 23:11:00.213508 5895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:11:00.216351 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5814
2026-09-28T23:11:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1287 ms)
[----------] 18 tests from AuthTokenIssuingTest (23319 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:11:00 dist-test-slave-x2t8 krb5kdc[5913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:11:00 dist-test-slave-x2t8 krb5kdc[5913](info): set up 2 sockets
Sep 28 23:11:00 dist-test-slave-x2t8 krb5kdc[5913](info): commencing operation
krb5kdc: starting...
W20260928 23:11:01.245622 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:11:01.368124 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34831
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:11:01.446810 5926 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:01.446998 5926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:01.447041 5926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:01.447060 5926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:01.448105 5926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:01.448136 5926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:01.448150 5926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:01.448163 5926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:01.448175 5926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:01.449534 5926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:34831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:01.449724 5926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:01.449890 5926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:11:01.454675 5926 server_base.cc:1034] running on GCE node
W20260928 23:11:01.454665 5932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:01.454638 5934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:01.454632 5931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:01.455087 5926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:01.455408 5926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:01.456573 5926 hybrid_clock.cc:648] HybridClock initialized: now 1790637061456555 us; error 31 us; skew 500 ppm
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:01.458788 5926 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:11:01.459254 5926 webserver.cc:460] Webserver started at http://127.15.212.126:39025/ using document root <none> and password file <none>
I20260928 23:11:01.459486 5926 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:01.459554 5926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:01.459712 5926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:01.460785 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "e40549160b54463484aa122769f86630"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "66244fb69e79e2faa5a6c66a1ee0fea6"
server_key_iv: "b3b9a4938181bea4d059055e967e141b"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.461125 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "e40549160b54463484aa122769f86630"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "66244fb69e79e2faa5a6c66a1ee0fea6"
server_key_iv: "b3b9a4938181bea4d059055e967e141b"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.462378 5926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 23:11:01.463029 5941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:01.463203 5926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:11:01.463304 5926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "e40549160b54463484aa122769f86630"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "66244fb69e79e2faa5a6c66a1ee0fea6"
server_key_iv: "b3b9a4938181bea4d059055e967e141b"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.463387 5926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:01.469905 5926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:01.471515 5926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:01.471668 5926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:01.475462 5926 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:34831
I20260928 23:11:01.475490 5993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:34831 every 8 connection(s)
I20260928 23:11:01.475800 5926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:11:01.476619 5994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:01.477933 5994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.482064 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 5926
I20260928 23:11:01.482190 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:11:01.482410 16209 external_mini_cluster.cc:1428] Setting key 4c0e659cb453c8d08f8cec4034cad48c
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:11:01.487260 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.478555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.15.212.126:34831)
0928 23:11:01.478712 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.478721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.478841 (+ 120us) server_negotiation.cc:407] Connection header received
0928 23:11:01.479410 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.479412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.479428 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.479498 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.480005 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.480858 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.481676 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.481974 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.482055 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.484178 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.484184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.484185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.484191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.485737 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.486177 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.486179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.486180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.486250 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.486509 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.486510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.486510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.486681 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.486737 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.487015 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.487121 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260928 23:11:01.487421 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.478476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34831 (local address 127.0.0.1:54338)
0928 23:11:01.478702 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.478743 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.479218 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.479537 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.479539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.479771 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.479925 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.479929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.480949 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.480950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.481604 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.481605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.482065 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.482066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.482162 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.482660 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.482667 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.484080 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.485813 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.485820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.485822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.486091 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.486316 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.486316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.486317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.486440 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.486751 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.486758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.486845 (+ 87us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.486986 (+ 141us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.487227 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":81,"thread_start_us":101,"threads_started":1}
I20260928 23:11:01.489204 5994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.489542 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.489412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34386)
0928 23:11:01.489468 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.489518 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 23:11:01.489686 5943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
I20260928 23:11:01.490098 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.490022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34396)
0928 23:11:01.490059 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.490082 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
WARNING: no policy specified for kudu/127.15.212.125@KRBTEST.COM; defaulting to no policy
W20260928 23:11:01.490212 5994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:43167: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
I20260928 23:11:01.507339 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:43167
--webserver_interface=127.15.212.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:11:01.539408 5994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } attempt: 1
I20260928 23:11:01.539898 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.539769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34410)
0928 23:11:01.539830 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.539870 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 23:11:01.540275 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.540163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34416)
0928 23:11:01.540220 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.540250 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 23:11:01.540378 5994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:43167: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
W20260928 23:11:01.586536 6001 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:01.586725 6001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:01.586742 6001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:01.586761 6001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:01.587950 6001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:01.587989 6001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:01.588003 6001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:01.588016 6001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:01.588048 6001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:01.589531 6001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.212.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:01.589769 6001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:01.589993 6001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:11:01.594769 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:01.594743 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:01.594738 6007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:01.594779 6001 server_base.cc:1034] running on GCE node
I20260928 23:11:01.595301 6001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:01.595609 6001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:01.596777 6001 hybrid_clock.cc:648] HybridClock initialized: now 1790637061596770 us; error 37 us; skew 500 ppm
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:01.599392 6001 init.cc:377] Logged in from keytab as kudu/127.15.212.125@KRBTEST.COM (short username kudu)
I20260928 23:11:01.599823 6001 webserver.cc:460] Webserver started at http://127.15.212.125:45615/ using document root <none> and password file <none>
I20260928 23:11:01.600082 6001 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:01.600157 6001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:01.600306 6001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:01.601493 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data/instance:
uuid: "ce1e5ef8b04147b8b43cd1b51113c87c"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "9d64dcdd395b2fdbc17a08e4f8509cf1"
server_key_iv: "c491b92ce853d866a64406b8bed499a0"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.601878 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal/instance:
uuid: "ce1e5ef8b04147b8b43cd1b51113c87c"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "9d64dcdd395b2fdbc17a08e4f8509cf1"
server_key_iv: "c491b92ce853d866a64406b8bed499a0"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.603255 6001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:11:01.604055 6016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:01.604244 6001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 23:11:01.604349 6001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "ce1e5ef8b04147b8b43cd1b51113c87c"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "9d64dcdd395b2fdbc17a08e4f8509cf1"
server_key_iv: "c491b92ce853d866a64406b8bed499a0"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.604434 6001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:01.608522 5994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } attempt: 2
I20260928 23:11:01.608899 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.608775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34428)
0928 23:11:01.608834 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.608870 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:11:01.609198 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.609108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34430)
0928 23:11:01.609152 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.609175 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:11:01.609293 5994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:43167: Network error: Client connection negotiation failed: client connection to 127.15.212.125:43167: connect: Connection refused (error 111)
I20260928 23:11:01.619083 6001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:01.621634 6001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:01.621804 6001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:01.626094 6001 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.125:43167
I20260928 23:11:01.626124 6068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.125:43167 every 8 connection(s)
I20260928 23:11:01.626479 6001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
I20260928 23:11:01.627445 6069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:01.629278 6069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.631590 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 6001
I20260928 23:11:01.631693 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-1/wal/instance
I20260928 23:11:01.631880 16209 external_mini_cluster.cc:1428] Setting key b74ef6f7137105f1eb5022ced27ab6db
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:11:01.638721 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.630113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.15.212.126:34831)
0928 23:11:01.630221 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.630224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.630393 (+ 169us) server_negotiation.cc:407] Connection header received
0928 23:11:01.630919 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.630920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.630933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.630991 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.631590 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.632189 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.633401 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.633613 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.633685 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.635907 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.635908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.635908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.635916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.637368 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.637791 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.637792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.637792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.637850 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.638101 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.638101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.638102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.638209 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.638273 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.638546 (+ 273us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.638634 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.15.212.124@KRBTEST.COM; defaulting to no policy
I20260928 23:11:01.638878 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.630092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34831 (local address 127.0.0.1:54342)
0928 23:11:01.630273 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.630305 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.630796 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.631058 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.631061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.631285 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.631494 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.631498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.632281 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.632282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.633292 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.633295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.633709 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.633710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.633854 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.634309 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.634318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.635812 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.637477 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.637484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.637486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.637707 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.637914 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.637915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.637915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.638022 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.638277 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.638283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.638412 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.638510 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.638670 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260928 23:11:01.639400 6069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.124@KRBTEST.COM" created.
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:01.646868 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.639649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34432)
0928 23:11:01.639689 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.639699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.639730 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.639967 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.639967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.640096 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.640134 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.640135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.641047 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.641048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.641583 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.641598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.642049 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.642050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.642111 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.642500 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.642504 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.643873 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.645730 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.645731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.645731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.645913 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.646140 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.646141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.646141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.646196 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.646493 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.646494 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.646532 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.646648 (+ 116us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.646754 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 23:11:01.646967 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.639684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.15.212.125:43167)
0928 23:11:01.639823 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.639827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.639850 (+ 23us) server_negotiation.cc:407] Connection header received
0928 23:11:01.639865 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.639867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.639876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.639966 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.640225 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.640946 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.641681 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.641962 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.642050 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.643970 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.643978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.643978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.643984 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.645624 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.646012 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.646015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.646015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.646071 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.646273 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.646274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.646274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.646413 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.646515 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.646697 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.646811 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260928 23:11:01.647495 6069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.647806 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.647716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45726)
0928 23:11:01.647747 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.647789 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260928 23:11:01.647946 6020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
I20260928 23:11:01.648362 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.648246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45734)
0928 23:11:01.648304 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.648338 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 23:11:01.648518 6069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33671: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
I20260928 23:11:01.655035 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:33671
--webserver_interface=127.15.212.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:43349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:11:01.697721 6069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } attempt: 1
I20260928 23:11:01.698174 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.698046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45736)
0928 23:11:01.698108 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.698145 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 23:11:01.698508 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.698416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45740)
0928 23:11:01.698462 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.698485 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 23:11:01.698596 6069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33671: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
I20260928 23:11:01.700462 5994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } attempt: 3
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:01.707512 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.700733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34442)
0928 23:11:01.700777 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.700787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.700827 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.700973 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.700974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.701151 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.701225 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.701227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.701839 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.701840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.702659 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.702662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.703015 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.703017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.703092 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.703500 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.703506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.704814 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.706468 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.706469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.706470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.706672 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.706885 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.706886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.706886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.706945 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.707209 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.707210 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.707258 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.707329 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.707419 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 23:11:01.707508 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.700774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.15.212.125:43167)
0928 23:11:01.700807 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.700809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.700819 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:11:01.700880 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.700881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.700889 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.700939 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.701297 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.701764 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.702740 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.702946 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.702995 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.704893 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.704895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.704895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.704903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.706385 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.706744 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.706745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.706746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.706805 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.707019 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.707020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.707021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.707133 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.707208 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.707353 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.707423 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 23:11:01.707916 5994 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.708272 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.708172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45754)
0928 23:11:01.708222 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.708247 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:11:01.708647 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.708564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45762)
0928 23:11:01.708611 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.708634 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 23:11:01.708755 5994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33671: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
W20260928 23:11:01.735535 6076 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:01.735723 6076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:01.735739 6076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:01.735759 6076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:01.736891 6076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:01.736929 6076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:01.736943 6076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:01.736956 6076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:01.736968 6076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:01.738327 6076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:43349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:34831,127.15.212.125:43167,127.15.212.124:33671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.212.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:01.738564 6076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:01.738781 6076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 23:11:01.739892 5994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } attempt: 1
I20260928 23:11:01.740300 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.740184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45772)
0928 23:11:01.740239 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.740276 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 23:11:01.740644 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.740551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45788)
0928 23:11:01.740593 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.740616 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 23:11:01.740732 5994 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33671: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
W20260928 23:11:01.743578 6084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:01.743716 6076 server_base.cc:1034] running on GCE node
W20260928 23:11:01.743799 6082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:01.743605 6081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:01.744063 6076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:01.744365 6076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:01.745504 6076 hybrid_clock.cc:648] HybridClock initialized: now 1790637061745481 us; error 38 us; skew 500 ppm
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:01.747761 6076 init.cc:377] Logged in from keytab as kudu/127.15.212.124@KRBTEST.COM (short username kudu)
I20260928 23:11:01.748202 6076 webserver.cc:460] Webserver started at http://127.15.212.124:34423/ using document root <none> and password file <none>
I20260928 23:11:01.748416 6076 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:01.748528 6076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:01.748679 6076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:01.749681 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data/instance:
uuid: "4cdd9ed804654a73a3417c54988e45f6"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "96711ccc0de45138f3df89157fee6682"
server_key_iv: "e67b04e2274e51ca0c57c03fd6451687"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.750033 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal/instance:
uuid: "4cdd9ed804654a73a3417c54988e45f6"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "96711ccc0de45138f3df89157fee6682"
server_key_iv: "e67b04e2274e51ca0c57c03fd6451687"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.751258 6076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:11:01.751919 6091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:01.752091 6076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 23:11:01.752183 6076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "4cdd9ed804654a73a3417c54988e45f6"
format_stamp: "Formatted at 2026-09-28 23:11:01 on dist-test-slave-x2t8"
server_key: "96711ccc0de45138f3df89157fee6682"
server_key_iv: "e67b04e2274e51ca0c57c03fd6451687"
server_key_version: "encryptionkey@0"
I20260928 23:11:01.752266 6076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:01.766724 6069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } attempt: 2
I20260928 23:11:01.766934 6076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:01.767110 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.766969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45792)
0928 23:11:01.767017 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.767078 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260928 23:11:01.767645 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.767546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45802)
0928 23:11:01.767589 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.767614 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 23:11:01.767729 6069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33671: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33671: connect: Connection refused (error 111)
I20260928 23:11:01.768784 6076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:01.768936 6076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:01.772616 6076 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.124:33671
I20260928 23:11:01.772643 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.124:33671 every 8 connection(s)
I20260928 23:11:01.772964 6076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
I20260928 23:11:01.773713 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:01.775028 6144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:01.778932 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 6076
I20260928 23:11:01.779034 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790636798761935-16209-0/minicluster-data/master-2/wal/instance
I20260928 23:11:01.779239 16209 external_mini_cluster.cc:1428] Setting key bc5b36e627ce7b12d9f5a33f55c44ca8
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:11:01.783224 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.775611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.15.212.126:34831)
0928 23:11:01.775670 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.775672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.775858 (+ 186us) server_negotiation.cc:407] Connection header received
0928 23:11:01.776270 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.776271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.776279 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.776332 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.776901 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.777369 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.778231 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.778401 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.778449 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.780648 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.780650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.780650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.780657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.781896 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.782307 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.782308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.782308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.782367 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.782617 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.782618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.782618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.782726 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.782800 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.783059 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.783140 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 23:11:01.783389 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.775589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:34831 (local address 127.0.0.1:54354)
0928 23:11:01.775745 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.775781 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.776194 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.776377 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.776380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.776658 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.776830 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.776835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.777440 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.777440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.778166 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.778168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.778464 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.778465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.778562 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.779040 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.779051 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.780563 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.781998 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.782005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.782007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.782218 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.782437 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.782437 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.782438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.782548 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.782792 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.782798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.782930 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.783037 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.783166 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260928 23:11:01.783900 6144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:01.787914 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.779622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.15.212.124:33671)
0928 23:11:01.779802 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.779807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.779830 (+ 23us) server_negotiation.cc:407] Connection header received
0928 23:11:01.779875 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.779878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.779896 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.779982 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.780662 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.781586 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.782323 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.782621 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.782696 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.785078 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.785084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.785085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.785091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.786753 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.787108 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.787111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.787111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.787167 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.787369 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.787370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.787370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.787513 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.787676 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.787719 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.787793 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260928 23:11:01.790350 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.784200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.15.212.125:43167)
0928 23:11:01.784250 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.784252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.784261 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:01.784282 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.784283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.784292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.784359 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.784662 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.785145 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.785786 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.786007 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.786088 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.787685 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.787686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.787687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.787693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.789375 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.789681 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.789682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.789682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.789739 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.789927 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.789928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.789928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.790040 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.790088 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.790239 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.790295 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:11:01.790390 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.784119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:43167 (local address 127.0.0.1:34448)
0928 23:11:01.784163 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.784173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.784208 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.784360 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.784361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.784524 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.784576 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.784577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.785222 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.785224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.785704 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.785705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.786089 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.786090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.786155 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.786494 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.786498 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.787601 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.789465 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.789466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.789466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.789616 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.789801 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.789802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.789802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.789848 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.790104 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.790105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.790145 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.790222 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.790301 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 23:11:01.791023 6144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:01.796049 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.788632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.15.212.126:34831)
0928 23:11:01.788665 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.788667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.788692 (+ 25us) server_negotiation.cc:407] Connection header received
0928 23:11:01.788710 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.788711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.788726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.788784 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.789220 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.789833 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.790688 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.790929 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.791003 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.793375 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.793376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.793376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.793385 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.794977 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.795296 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.795297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.795297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.795347 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.795554 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.795555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.795556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.795662 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.795840 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.795875 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.795956 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:11:01.797578 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.791233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45828)
0928 23:11:01.791270 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.791278 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.791309 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.791442 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.791442 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.791588 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.791621 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.791623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.792191 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.792192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.792772 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.792774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.793255 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.793255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.793290 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.793658 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.793662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.794855 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.796505 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.796506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.796507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.796723 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.797031 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.797032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.797033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.797100 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.797301 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.797302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.797355 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.797434 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.797515 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260928 23:11:01.798161 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.791276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.15.212.124:33671)
0928 23:11:01.791307 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.791308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.791316 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:11:01.791364 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.791365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.791374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.791437 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.791697 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.792117 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.792844 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.793180 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.793255 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.794932 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.794933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.794933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.794938 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.796409 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.796919 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.796919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.796920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.796966 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.797167 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.797167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.797167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.797245 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.797286 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.798054 (+ 768us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.798112 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 23:11:01.799372 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: Bootstrap starting.
I20260928 23:11:01.800470 6144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:01.800772 6144 log.cc:824] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: Log is configured to *not* fsync() on all Append() calls
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:01.801489 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: No bootstrap required, opened a new log
I20260928 23:11:01.803092 6144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.803229 6144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdd9ed804654a73a3417c54988e45f6, State: Initialized, Role: FOLLOWER
I20260928 23:11:01.803344 6144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.803628 6144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:01.803610 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.803714 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.804116 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.796774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.15.212.125:43167)
0928 23:11:01.796809 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.796810 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.796819 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:01.796836 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.796837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.796844 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.796894 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.797467 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.797967 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.798787 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.799003 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.799072 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.801574 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.801576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.801576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.801583 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.802868 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.803226 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.803226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.803227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.803279 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.803474 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.803475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.803476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.803596 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.803746 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.803935 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.804007 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 23:11:01.805505 6161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:11:01.805560 6161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:11:01.805876 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:11:01.806953 5968 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e40549160b54463484aa122769f86630"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54362
I20260928 23:11:01.805701 6076 master_runner.cc:186] Error getting master registration for 127.15.212.126:34831: OK, instance_id { permanent_uuid: "e40549160b54463484aa122769f86630" instance_seqno: 1790637061474402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:11:01.808426 6076 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:11:01.815871 5994 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } attempt: 2
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:01.823453 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.816129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45844)
0928 23:11:01.816175 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.816185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.816222 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.816359 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.816360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.816535 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.816575 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.816577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.817452 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.817453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.818259 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.818262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.818700 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.818701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.818763 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.819169 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.819174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.820589 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.822378 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.822379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.822380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.822595 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.822792 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.822793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.822794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.822863 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.823153 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.823154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.823198 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.823270 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.823355 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 23:11:01.823462 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.816208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.15.212.124:33671)
0928 23:11:01.816242 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.816244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.816254 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:11:01.816273 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.816274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.816282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.816359 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.816651 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.817368 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.818348 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.818632 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.818687 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.820698 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.820699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.820699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.820706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.822295 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.822669 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.822670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.822670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.822726 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.822931 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.822932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.822932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.823084 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.823151 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.823300 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.823374 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 23:11:01.825201 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: Bootstrap starting.
I20260928 23:11:01.825901 5994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:01.826184 5994 log.cc:824] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:01.826923 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: No bootstrap required, opened a new log
I20260928 23:11:01.828222 5994 raft_consensus.cc:348] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.828353 5994 raft_consensus.cc:729] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40549160b54463484aa122769f86630, State: Initialized, Role: FOLLOWER
I20260928 23:11:01.828428 5994 consensus_queue.cc:260] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.828742 5994 sys_catalog.cc:564] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:01.828760 5968 raft_consensus.cc:482] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 23:11:01.828809 5968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:01.829607 5968 raft_consensus.cc:504] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.830137 6118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e40549160b54463484aa122769f86630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4cdd9ed804654a73a3417c54988e45f6"
I20260928 23:11:01.830255 6118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:01.830204 6164 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.830322 6164 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.830947 6118 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40549160b54463484aa122769f86630 in term 1.
I20260928 23:11:01.831221 5945 leader_election.cc:302] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cdd9ed804654a73a3417c54988e45f6, e40549160b54463484aa122769f86630; no voters:
I20260928 23:11:01.831347 5968 leader_election.cc:288] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [CANDIDATE]: Term 1 election: Requested vote from peers ce1e5ef8b04147b8b43cd1b51113c87c (127.15.212.125:43167), 4cdd9ed804654a73a3417c54988e45f6 (127.15.212.124:33671)
I20260928 23:11:01.831339 6164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:11:01.831552 6164 raft_consensus.cc:686] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [term 1 LEADER]: Becoming Leader. State: Replica: e40549160b54463484aa122769f86630, State: Running, Role: LEADER
I20260928 23:11:01.831624 6164 consensus_queue.cc:237] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.832027 6164 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e40549160b54463484aa122769f86630. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.832088 6164 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: This master's current role is: LEADER
W20260928 23:11:01.832708 6175 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:11:01.832756 6175 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:11:01.834154 6181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:11:01.834334 6181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:11:01.832976 5926 master_runner.cc:186] Error getting master registration for 127.15.212.125:43167: OK, instance_id { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" instance_seqno: 1790637061624861 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:11:01.835157 5926 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:11:01.837170 6118 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [term 1 FOLLOWER]: Refusing update from remote peer e40549160b54463484aa122769f86630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:11:01.837584 6176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:11:01.839429 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e40549160b54463484aa122769f86630. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.839520 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.849625 6176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.849718 6176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:01.850180 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.850265 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.850322 6187 mvcc.cc:205] Tried to move back new op lower bound from 7334449405281304576 to 7334449405262962688. Current Snapshot: MvccSnapshot[applied={T|T < 7334449405281304576}]
I20260928 23:11:01.850382 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.833722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.15.212.124:33671)
0928 23:11:01.833752 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.833753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.834804 (+ 1051us) server_negotiation.cc:407] Connection header received
0928 23:11:01.835150 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.835151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.835161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.835188 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.837458 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.838131 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.843231 (+ 5100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.843512 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.843555 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.846377 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.846379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.846379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.846386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.848483 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.849095 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.849097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.849097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.849151 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.849462 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.849463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.849463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.849598 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.849689 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.850142 (+ 453us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.850219 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 23:11:01.850428 6188 mvcc.cc:205] Tried to move back new op lower bound from 7334449405281304576 to 7334449405262962688. Current Snapshot: MvccSnapshot[applied={T|T < 7334449405281304576}]
I20260928 23:11:01.851893 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.833350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.15.212.125:43167)
0928 23:11:01.833388 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.833391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.834887 (+ 1496us) server_negotiation.cc:407] Connection header received
0928 23:11:01.836655 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.836657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.836669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.836700 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.841535 (+ 4835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.842250 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.843957 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.844262 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.844303 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.846881 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.846882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.846882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.846889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.849251 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.850685 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.850686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.850687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.850743 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.851344 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.851345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.851345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.851470 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.851547 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.851727 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.851803 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:11:01.855223 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.833529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.15.212.126:34831)
0928 23:11:01.833564 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.833566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.834712 (+ 1146us) server_negotiation.cc:407] Connection header received
0928 23:11:01.835053 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.835054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.835064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.835093 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.838448 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.839159 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.844632 (+ 5473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.844931 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.844972 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.852303 (+ 7331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.852304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.852304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.852312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.853929 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.854388 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.854389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.854389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.854442 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.854628 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.854629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.854629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.854741 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.854827 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.855041 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.855112 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:11:01.856070 6181 catalog_manager.cc:1347] Generated new cluster ID: d2ee09871e774c5082d503a8d560b40f
I20260928 23:11:01.856117 6181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:11:01.858841 6069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } attempt: 3
Sep 28 23:11:01 dist-test-slave-x2t8 krb5kdc[5913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:01.868644 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.860692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33671 (local address 127.0.0.1:45856)
0928 23:11:01.860772 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:01.860782 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:01.860817 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:01.860981 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:01.860982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:01.861131 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:01.861168 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.861170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.862014 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.862015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.862828 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:01.862830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.863348 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.863349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:01.863425 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.863885 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:01.863890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:01.865374 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:01.867482 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.867483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.867484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.867688 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.867945 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:01.867946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:01.867946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:01.868017 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:01.868325 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:01.868326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:01.868381 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 23:11:01.868458 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 23:11:01.868550 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 23:11:01.869058 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:01.860769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.15.212.124:33671)
0928 23:11:01.860816 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.860818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.860827 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:01.860891 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.860892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.860903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.860947 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.861240 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.861937 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.862935 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:01.863224 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:01.863282 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:01.865481 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:01.865482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:01.865482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:01.865489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:01.867286 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.867785 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.867786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.867787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.867854 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:01.868097 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:01.868098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:01.868098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:01.868229 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:01.868284 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:01.868892 (+ 608us) server_negotiation.cc:299] Negotiation successful
0928 23:11:01.868970 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 23:11:01.870736 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: Bootstrap starting.
I20260928 23:11:01.871429 6069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:01.871701 6069 log.cc:824] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:01.872701 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: No bootstrap required, opened a new log
I20260928 23:11:01.874660 6069 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.874804 6069 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1e5ef8b04147b8b43cd1b51113c87c, State: Initialized, Role: FOLLOWER
I20260928 23:11:01.874881 6069 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } }
I20260928 23:11:01.875173 6069 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:01.876391 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e40549160b54463484aa122769f86630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c"
I20260928 23:11:01.876614 6043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:01.877465 6043 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e40549160b54463484aa122769f86630 in term 1.
I20260928 23:11:01.877632 6042 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [term 1 FOLLOWER]: Refusing update from remote peer e40549160b54463484aa122769f86630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:11:01.878141 6164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 23:11:01.881228 6204 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:11:01.881321 6204 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:11:01.881690 6193 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.881826 6193 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.881910 6193 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e40549160b54463484aa122769f86630. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.881999 6193 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.882063 6193 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.882145 6193 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:01.882921 6206 mvcc.cc:205] Tried to move back new op lower bound from 7334449405281304576 to 7334449405262962688. Current Snapshot: MvccSnapshot[applied={T|T < 7334449405281304576}]
I20260928 23:11:01.888823 6164 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e40549160b54463484aa122769f86630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e40549160b54463484aa122769f86630" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 34831 } } peers { permanent_uuid: "ce1e5ef8b04147b8b43cd1b51113c87c" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 43167 } } peers { permanent_uuid: "4cdd9ed804654a73a3417c54988e45f6" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33671 } } }
I20260928 23:11:01.888928 6164 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:01.890054 6181 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:11:01.891331 6181 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:11:01.903352 6181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e40549160b54463484aa122769f86630: Generated new TSK 0
I20260928 23:11:01.903568 6181 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 23:11:01.932060 6148 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.930312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.15.212.124:33671)
0928 23:11:01.930359 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.930361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.931347 (+ 986us) server_negotiation.cc:407] Connection header received
0928 23:11:01.931458 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.931459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.931470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.931501 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.931994 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45866: BlockingRecv error: recv got EOF from 127.0.0.1:45866 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 23:11:01.933027 5997 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.930716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.15.212.126:34831)
0928 23:11:01.930758 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.930759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.932500 (+ 1741us) server_negotiation.cc:407] Connection header received
0928 23:11:01.932601 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.932602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.932613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.932643 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.932966 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54394: BlockingRecv error: recv got EOF from 127.0.0.1:54394 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 23:11:01.933965 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.931065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.15.212.125:43167)
0928 23:11:01.931105 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.931107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.933446 (+ 2339us) server_negotiation.cc:407] Connection header received
0928 23:11:01.933546 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.933547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.933558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.933588 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.933908 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34466: BlockingRecv error: recv got EOF from 127.0.0.1:34466 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 23:11:01.936478 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.935690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.15.212.125:43167)
0928 23:11:01.935724 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.935726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.935953 (+ 227us) server_negotiation.cc:407] Connection header received
0928 23:11:01.936056 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.936057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.936074 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.936103 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.936415 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34468: BlockingRecv error: recv got EOF from 127.0.0.1:34468 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 23:11:01.939450 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.938748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.15.212.125:43167)
0928 23:11:01.938778 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.938780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.939006 (+ 226us) server_negotiation.cc:407] Connection header received
0928 23:11:01.939096 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.939098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.939107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.939134 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.939412 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34476: BlockingRecv error: recv got EOF from 127.0.0.1:34476 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 23:11:01.943300 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.942594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.15.212.125:43167)
0928 23:11:01.942626 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.942628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.942828 (+ 200us) server_negotiation.cc:407] Connection header received
0928 23:11:01.942933 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.942934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.942943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.942970 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.943259 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34486: BlockingRecv error: recv got EOF from 127.0.0.1:34486 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 23:11:01.949193 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.948389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.15.212.125:43167)
0928 23:11:01.948420 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.948457 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.948679 (+ 222us) server_negotiation.cc:407] Connection header received
0928 23:11:01.948801 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.948802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.948811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.948839 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.949147 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34488: BlockingRecv error: recv got EOF from 127.0.0.1:34488 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 23:11:01.959149 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.958371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.15.212.125:43167)
0928 23:11:01.958405 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.958407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.958635 (+ 228us) server_negotiation.cc:407] Connection header received
0928 23:11:01.958730 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.958731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.958741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.958769 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.959094 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34502: BlockingRecv error: recv got EOF from 127.0.0.1:34502 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 23:11:01.977308 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:01.976486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34504 (local address 127.15.212.125:43167)
0928 23:11:01.976524 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:01.976526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:01.976784 (+ 258us) server_negotiation.cc:407] Connection header received
0928 23:11:01.976892 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:01.976893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:01.976904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:01.976931 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:01.977261 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34504: BlockingRecv error: recv got EOF from 127.0.0.1:34504 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 23:11:02.011332 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:02.010603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.15.212.125:43167)
0928 23:11:02.010648 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.010650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.010782 (+ 132us) server_negotiation.cc:407] Connection header received
0928 23:11:02.010856 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.010858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.010869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.010938 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:02.011259 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34516: BlockingRecv error: recv got EOF from 127.0.0.1:34516 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 23:11:02.077137 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:02.076485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.15.212.125:43167)
0928 23:11:02.076552 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.076554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.076657 (+ 103us) server_negotiation.cc:407] Connection header received
0928 23:11:02.076703 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.076704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.076713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.076776 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:02.077068 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34524: BlockingRecv error: recv got EOF from 127.0.0.1:34524 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 23:11:02.207059 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:02.206209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.15.212.125:43167)
0928 23:11:02.206288 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.206290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.206390 (+ 100us) server_negotiation.cc:407] Connection header received
0928 23:11:02.206489 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.206490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.206502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.206576 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:02.206971 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34536: BlockingRecv error: recv got EOF from 127.0.0.1:34536 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 23:11:02.465077 6073 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:02.464354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.15.212.125:43167)
0928 23:11:02.464457 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.464459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.464469 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:11:02.464572 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.464573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.464588 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.464662 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:02.464998 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34546: BlockingRecv error: recv got EOF from 127.0.0.1:34546 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 23:11:02.805950 6161 catalog_manager.cc:1259] Loaded cluster ID: d2ee09871e774c5082d503a8d560b40f
I20260928 23:11:02.806021 6161 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: loading cluster ID for follower catalog manager: success
I20260928 23:11:02.806847 6161 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: acquiring CA information for follower catalog manager: success
I20260928 23:11:02.807170 6161 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4cdd9ed804654a73a3417c54988e45f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:11:02.881795 6204 catalog_manager.cc:1259] Loaded cluster ID: d2ee09871e774c5082d503a8d560b40f
I20260928 23:11:02.881862 6204 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: loading cluster ID for follower catalog manager: success
I20260928 23:11:02.882586 6204 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: acquiring CA information for follower catalog manager: success
I20260928 23:11:02.882834 6204 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce1e5ef8b04147b8b43cd1b51113c87c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:11:02.980659 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:02.978252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34552 (local address 127.15.212.125:43167)
0928 23:11:02.978365 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.978366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.978378 (+ 12us) server_negotiation.cc:407] Connection header received
0928 23:11:02.978461 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.978462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.978509 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.978583 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:02.978725 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.979213 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.979924 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.980112 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.980159 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:02.980303 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:02.980406 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:02.980478 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:02.980542 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 23:11:02.980558 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260928 23:11:02.984956 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:02.982505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.15.212.124:33671)
0928 23:11:02.982616 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.982618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.982627 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:02.982704 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.982705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.982746 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.982838 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:02.982959 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.983436 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.984151 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.984361 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.984474 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:02.984626 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:02.984729 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:02.984811 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:02.984851 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 23:11:02.984871 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260928 23:11:02.989174 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:02.986615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.15.212.125:43167)
0928 23:11:02.986646 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.986648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.986655 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:11:02.986727 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.986727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.986752 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.986839 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:02.986962 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.987543 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.988360 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.988630 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.988702 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:02.988891 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:02.988920 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:02.988946 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:02.989072 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 23:11:02.989092 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 23:11:02.989423 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:02.986538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.15.212.124:33671)
0928 23:11:02.986577 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.986579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.986763 (+ 184us) server_negotiation.cc:407] Connection header received
0928 23:11:02.986788 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.986788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.986797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.986842 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:02.987017 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.987615 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.988687 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.988961 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.989025 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:02.989182 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:02.989212 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:02.989238 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:02.989330 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 23:11:02.989351 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:11:02.990352 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:02.986449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.15.212.126:34831)
0928 23:11:02.986847 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:02.986848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:02.986855 (+ 7us) server_negotiation.cc:407] Connection header received
0928 23:11:02.986896 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:02.986897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:02.986944 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:02.987020 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:02.987849 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.988623 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.989686 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:02.989969 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:02.990023 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:02.990148 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:02.990215 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:02.990253 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:02.990280 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 23:11:02.990297 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":62,"threads_started":1}
I20260928 23:11:02.992300 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 5926
I20260928 23:11:02.997817 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 6001
I20260928 23:11:03.002508 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 6076
2026-09-28T23:11:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2788 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 23:11:03 dist-test-slave-x2t8 krb5kdc[6292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 23:11:03 dist-test-slave-x2t8 krb5kdc[6292](info): set up 2 sockets
Sep 28 23:11:03 dist-test-slave-x2t8 krb5kdc[6292](info): commencing operation
krb5kdc: starting...
W20260928 23:11:04.024871 16209 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/test-user.keytab.
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T23:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T23:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.212.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
Entry for principal kudu/127.15.212.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab.
I20260928 23:11:04.138527 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37187
--webserver_interface=127.15.212.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 23:11:04.216591 6305 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:04.216778 6305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:04.216795 6305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:04.216841 6305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:04.218004 6305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:04.218043 6305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:04.218057 6305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:04.218097 6305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:04.218125 6305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:04.219519 6305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.126.keytab
--principal=kudu/127.15.212.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.126:37187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.212.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:04.219758 6305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:04.219973 6305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:11:04.224650 6311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.224682 6305 server_base.cc:1034] running on GCE node
W20260928 23:11:04.224654 6310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:04.224668 6313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.225131 6305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:04.225438 6305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:04.226603 6305 hybrid_clock.cc:648] HybridClock initialized: now 1790637064226598 us; error 35 us; skew 500 ppm
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:04.228941 6305 init.cc:377] Logged in from keytab as kudu/127.15.212.126@KRBTEST.COM (short username kudu)
I20260928 23:11:04.229331 6305 webserver.cc:460] Webserver started at http://127.15.212.126:45213/ using document root <none> and password file <none>
I20260928 23:11:04.229540 6305 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:04.229605 6305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:04.229750 6305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:04.230724 6305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data/instance:
uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "1223d1ef18f342e6170fce6ba4ada430"
server_key_iv: "7cacfb8d72d898a8642bccc5a985778d"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.231072 6305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance:
uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "1223d1ef18f342e6170fce6ba4ada430"
server_key_iv: "7cacfb8d72d898a8642bccc5a985778d"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.232319 6305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 23:11:04.233034 6320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:04.233182 6305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:04.233276 6305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal
uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "1223d1ef18f342e6170fce6ba4ada430"
server_key_iv: "7cacfb8d72d898a8642bccc5a985778d"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.233358 6305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:04.242906 6305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:04.245069 6305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:04.245221 6305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:04.248839 6305 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.126:37187
I20260928 23:11:04.248884 6372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.126:37187 every 8 connection(s)
I20260928 23:11:04.249218 6305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/data/info.pb
I20260928 23:11:04.250046 6373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:04.251452 6373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:04.252642 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 6305
I20260928 23:11:04.252722 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-0/wal/instance
I20260928 23:11:04.252905 16209 external_mini_cluster.cc:1428] Setting key 3809fbc532d968cc3d25e4418e878e1a
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.212.125@KRBTEST.COM; defaulting to no policy
I20260928 23:11:04.261276 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.252106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.15.212.126:37187)
0928 23:11:04.252247 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.252254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.252410 (+ 156us) server_negotiation.cc:407] Connection header received
0928 23:11:04.253082 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.253085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.253117 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.253208 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.253684 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.254704 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.255536 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.255818 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.255933 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.258165 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.258171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.258171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.258178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.259684 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.260081 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.260084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.260084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.260192 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.260403 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.260404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.260404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.260618 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.260729 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.260928 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.261059 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260928 23:11:04.261286 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.252080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37187 (local address 127.0.0.1:44836)
0928 23:11:04.252298 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.252323 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.252888 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.253227 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.253230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.253444 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.253592 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.253596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.254809 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.254810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.255437 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.255438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.255906 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.255907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.255967 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.256477 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.256486 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.258067 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.259779 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.259785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.259787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.259987 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.260254 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.260255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.260256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.260340 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.260696 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.260726 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.260784 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.260884 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.261118 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":104,"thread_start_us":90,"threads_started":1}
I20260928 23:11:04.262017 6373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:04.262360 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.262247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50034)
0928 23:11:04.262294 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.262333 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 23:11:04.262502 6322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.125@KRBTEST.COM" created.
I20260928 23:11:04.263015 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.262876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50048)
0928 23:11:04.262949 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.262981 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 23:11:04.263139 6373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:41971: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
Entry for principal kudu/127.15.212.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab.
I20260928 23:11:04.276907 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:41971
--webserver_interface=127.15.212.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:11:04.312371 6373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } attempt: 1
I20260928 23:11:04.312906 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.312757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50054)
0928 23:11:04.312823 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.312872 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 23:11:04.313275 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.313184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50070)
0928 23:11:04.313227 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.313255 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 23:11:04.313401 6373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:41971: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
W20260928 23:11:04.355837 6380 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:04.356027 6380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:04.356045 6380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:04.356092 6380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:04.357306 6380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:04.357366 6380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:04.357380 6380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:04.357432 6380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:04.357455 6380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:04.358927 6380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.125.keytab
--principal=kudu/127.15.212.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.125:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.212.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:04.359170 6380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:04.359401 6380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:11:04.363924 6386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:04.363902 6388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.364087 6380 server_base.cc:1034] running on GCE node
W20260928 23:11:04.363962 6385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.364351 6380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:04.364666 6380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:04.365816 6380 hybrid_clock.cc:648] HybridClock initialized: now 1790637064365796 us; error 40 us; skew 500 ppm
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:04.368036 6380 init.cc:377] Logged in from keytab as kudu/127.15.212.125@KRBTEST.COM (short username kudu)
I20260928 23:11:04.368425 6380 webserver.cc:460] Webserver started at http://127.15.212.125:46677/ using document root <none> and password file <none>
I20260928 23:11:04.368687 6380 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:04.368754 6380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:04.368897 6380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:04.369892 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data/instance:
uuid: "edad5c2b2ac940929c4e07bbb19ba53a"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "cb3bd80565611ddd05adfa6201ba2a1d"
server_key_iv: "9661ea55208af034cb5e21853cc0ff8e"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.370239 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal/instance:
uuid: "edad5c2b2ac940929c4e07bbb19ba53a"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "cb3bd80565611ddd05adfa6201ba2a1d"
server_key_iv: "9661ea55208af034cb5e21853cc0ff8e"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.371531 6380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 23:11:04.372217 6395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:04.372380 6380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 23:11:04.372505 6380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal
uuid: "edad5c2b2ac940929c4e07bbb19ba53a"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "cb3bd80565611ddd05adfa6201ba2a1d"
server_key_iv: "9661ea55208af034cb5e21853cc0ff8e"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.372589 6380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:04.381574 6373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } attempt: 2
I20260928 23:11:04.381956 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.381834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50086)
0928 23:11:04.381891 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.381928 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:11:04.382279 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.382202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50094)
0928 23:11:04.382242 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.382264 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 23:11:04.382392 6373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.125:41971: Network error: Client connection negotiation failed: client connection to 127.15.212.125:41971: connect: Connection refused (error 111)
I20260928 23:11:04.413396 6380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:04.415220 6380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:04.415366 6380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:04.419176 6380 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.125:41971
I20260928 23:11:04.419219 6447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.125:41971 every 8 connection(s)
I20260928 23:11:04.419589 6380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/data/info.pb
I20260928 23:11:04.420401 6448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:04.421584 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 6380
I20260928 23:11:04.421690 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-1/wal/instance
I20260928 23:11:04.421898 16209 external_mini_cluster.cc:1428] Setting key e111f22f4f4b37f72f87d0482b900037
I20260928 23:11:04.422304 6448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.212.124@KRBTEST.COM; defaulting to no policy
I20260928 23:11:04.432183 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.423155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.15.212.126:37187)
0928 23:11:04.423341 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.423342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.423523 (+ 181us) server_negotiation.cc:407] Connection header received
0928 23:11:04.423985 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.423986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.424008 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.424084 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.424731 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.425243 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.426743 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.426940 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.426994 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.429343 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.429345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.429345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.429357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.430770 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.431219 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.431221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.431222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.431283 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.431542 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.431543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.431543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.431658 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.431736 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.431998 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.432086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260928 23:11:04.432416 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.423124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37187 (local address 127.0.0.1:44850)
0928 23:11:04.423408 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.423438 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.423877 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.424155 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.424158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.424390 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.424651 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.424656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.425339 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.425340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.426660 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.426664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.427030 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.427031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.427184 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.427658 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.427676 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.429234 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.430867 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.430874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.430876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.431098 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.431353 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.431353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.431354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.431464 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.431728 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.431734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.431871 (+ 137us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.431977 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.432199 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.212.124@KRBTEST.COM" created.
I20260928 23:11:04.433028 6448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:04.440178 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.433281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50100)
0928 23:11:04.433332 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.433343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.433383 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.433636 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.433636 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.433770 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.433810 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.433811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.434642 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.434643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.435341 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.435343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.435804 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.435805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.435870 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.436279 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.436284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.437511 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.439139 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.439140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.439141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.439300 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.439514 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.439515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.439516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.439570 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.439854 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.439855 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.439891 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.439952 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.440049 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 23:11:04.440241 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.433354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.15.212.125:41971)
0928 23:11:04.433491 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.433496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.433528 (+ 32us) server_negotiation.cc:407] Connection header received
0928 23:11:04.433545 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.433546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.433556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.433635 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.433890 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.434552 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.435407 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.435709 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.435795 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.437606 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.437613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.437613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.437619 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.439043 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.439378 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.439381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.439381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.439436 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.439640 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.439641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.439641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.439783 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.439876 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.439989 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.440100 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260928 23:11:04.440879 6448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:04.441201 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.441106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54010)
0928 23:11:04.441142 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.441183 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260928 23:11:04.441342 6397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
I20260928 23:11:04.441713 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.441617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54026)
0928 23:11:04.441667 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.441691 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 23:11:04.441849 6448 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33987: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
Entry for principal kudu/127.15.212.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab.
I20260928 23:11:04.447965 16209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR8rxt/build/release/bin/kudu
/tmp/dist-test-taskdR8rxt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:33987
--webserver_interface=127.15.212.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.212.84:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 23:11:04.473558 6373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } attempt: 3
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:04.480592 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.473978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.15.212.125:41971)
0928 23:11:04.474052 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.474054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.474065 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:11:04.474113 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.474114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.474126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.474197 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.474523 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.475133 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.475849 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.476142 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.476224 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.478120 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.478121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.478122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.478131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.479418 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.479772 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.479773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.479774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.479830 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.480051 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.480052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.480053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.480163 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.480237 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.480407 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.480511 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 23:11:04.480587 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.473894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50106)
0928 23:11:04.473957 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.473972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.474049 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.474202 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.474204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.474367 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.474429 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.474431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.475226 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.475227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.475756 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.475758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.476222 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.476223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.476308 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.476809 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.476813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.477993 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.479504 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.479506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.479506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.479684 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.479907 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.479908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.479908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.479972 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.480242 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.480243 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.480286 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.480367 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.480484 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 23:11:04.481050 6373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } has no permanent_uuid. Determining permanent_uuid...
I20260928 23:11:04.481388 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.481285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54032)
0928 23:11:04.481334 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.481362 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 23:11:04.481694 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.481600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54044)
0928 23:11:04.481650 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.481673 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 23:11:04.481784 6373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33987: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
I20260928 23:11:04.491067 6448 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } attempt: 1
I20260928 23:11:04.491442 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.491336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54050)
0928 23:11:04.491390 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.491422 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:11:04.491792 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.491696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54060)
0928 23:11:04.491745 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.491770 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 23:11:04.491882 6448 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33987: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
I20260928 23:11:04.512917 6373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } attempt: 1
I20260928 23:11:04.513319 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.513198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54068)
0928 23:11:04.513255 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.513292 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 23:11:04.513641 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.513545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54072)
0928 23:11:04.513593 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.513619 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 23:11:04.513722 6373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33987: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
W20260928 23:11:04.528296 6455 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 23:11:04.528561 6455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 23:11:04.528582 6455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 23:11:04.528640 6455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 23:11:04.529767 6455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 23:11:04.529803 6455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 23:11:04.529817 6455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 23:11:04.529870 6455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 23:11:04.529889 6455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 23:11:04.531306 6455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.212.84:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.212.126:37187,127.15.212.125:41971,127.15.212.124:33987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/krb5kdc/kudu_127.15.212.124.keytab
--principal=kudu/127.15.212.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.212.124:33987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.212.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 22:19:40 UTC on ccycloud.quasar-blkhea.root.comops.site
build id 4678931
I20260928 23:11:04.531543 6455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 23:11:04.531765 6455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 23:11:04.536631 6463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.536660 6455 server_base.cc:1034] running on GCE node
W20260928 23:11:04.536615 6461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 23:11:04.536631 6460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 23:11:04.537072 6455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 23:11:04.537418 6455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 23:11:04.538581 6455 hybrid_clock.cc:648] HybridClock initialized: now 1790637064538560 us; error 34 us; skew 500 ppm
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 23:11:04.540934 6455 init.cc:377] Logged in from keytab as kudu/127.15.212.124@KRBTEST.COM (short username kudu)
I20260928 23:11:04.541292 6455 webserver.cc:460] Webserver started at http://127.15.212.124:44507/ using document root <none> and password file <none>
I20260928 23:11:04.541468 6455 fs_manager.cc:359] Metadata directory not provided
I20260928 23:11:04.541514 6455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 23:11:04.541606 6455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 23:11:04.542619 6455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data/instance:
uuid: "7d7a5d0a5e514575addfdf8ac122b4f9"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "c3da4005e3ee3244f39abf1914eae7bd"
server_key_iv: "01d45e3f522b8fe06f395bd9af8c591a"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.542902 6455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal/instance:
uuid: "7d7a5d0a5e514575addfdf8ac122b4f9"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "c3da4005e3ee3244f39abf1914eae7bd"
server_key_iv: "01d45e3f522b8fe06f395bd9af8c591a"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.544143 6455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 23:11:04.544862 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:04.545012 6455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 23:11:04.545101 6455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal
uuid: "7d7a5d0a5e514575addfdf8ac122b4f9"
format_stamp: "Formatted at 2026-09-28 23:11:04 on dist-test-slave-x2t8"
server_key: "c3da4005e3ee3244f39abf1914eae7bd"
server_key_iv: "01d45e3f522b8fe06f395bd9af8c591a"
server_key_version: "encryptionkey@0"
I20260928 23:11:04.545153 6455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 23:11:04.558369 6455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 23:11:04.560021 6448 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } attempt: 2
I20260928 23:11:04.560421 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.560304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54080)
0928 23:11:04.560360 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.560395 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 23:11:04.560643 6455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 23:11:04.560823 6455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 23:11:04.560864 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.560758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54094)
0928 23:11:04.560812 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.560846 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 23:11:04.561046 6448 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.212.124:33987: Network error: Client connection negotiation failed: client connection to 127.15.212.124:33987: connect: Connection refused (error 111)
I20260928 23:11:04.564530 6455 rpc_server.cc:307] RPC server started. Bound to: 127.15.212.124:33987
I20260928 23:11:04.564569 6522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.212.124:33987 every 8 connection(s)
I20260928 23:11:04.564908 6455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/data/info.pb
I20260928 23:11:04.565732 6523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 23:11:04.567164 6523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
I20260928 23:11:04.572294 16209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR8rxt/build/release/bin/kudu as pid 6455
I20260928 23:11:04.572397 16209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR8rxt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790636798761935-16209-0/minicluster-data/master-2/wal/instance
I20260928 23:11:04.572614 16209 external_mini_cluster.cc:1428] Setting key e9f06a2fc9c4186ed9b095333ec0cd97
I20260928 23:11:04.575806 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.567817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.15.212.126:37187)
0928 23:11:04.567893 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.567895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.568065 (+ 170us) server_negotiation.cc:407] Connection header received
0928 23:11:04.568581 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.568582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.568592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.568634 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.569167 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.569644 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.570556 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.570728 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.570776 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.572864 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.572866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.572866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.572876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.574374 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.574796 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.574798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.574798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.574859 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.575166 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.575167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.575167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.575298 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.575376 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.575643 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.575721 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 23:11:04.576002 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.567770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.126:37187 (local address 127.0.0.1:44856)
0928 23:11:04.567939 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.567971 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.568439 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.568703 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.568706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.568917 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.569100 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.569103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.569714 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.569714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.570489 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.570491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.570795 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.570795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.570887 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.571338 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.571346 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.572770 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.574478 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.574485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.574487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.574715 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.574936 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.574937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.574937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.575048 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.575367 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.575374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.575516 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.575621 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.575760 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260928 23:11:04.576563 6523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:04.581388 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.572976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.15.212.124:33987)
0928 23:11:04.573132 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.573137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.573160 (+ 23us) server_negotiation.cc:407] Connection header received
0928 23:11:04.573213 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.573215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.573228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.573323 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.573886 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.574548 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.575394 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.575688 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.575766 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.578521 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.578528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.578528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.578533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.580211 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.580588 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.580591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.580591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.580649 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.580850 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.580851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.580852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.580984 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.581167 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.581209 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.581283 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260928 23:11:04.583699 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.576821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.125:41971 (local address 127.0.0.1:50122)
0928 23:11:04.576940 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.576951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.577006 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.577134 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.577135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.577282 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.577352 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.577354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.578217 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.578218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.578810 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.578811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.579199 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.579199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.579270 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.579705 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.579708 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.581014 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.582753 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.582754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.582754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.582957 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.583164 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.583165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.583165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.583214 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.583449 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.583450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.583478 (+ 28us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.583525 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.583615 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260928 23:11:04.583717 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.576890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.15.212.125:41971)
0928 23:11:04.576942 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.576944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.576990 (+ 46us) server_negotiation.cc:407] Connection header received
0928 23:11:04.577079 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.577080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.577088 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.577130 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.577433 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.578145 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.578900 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.579118 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.579189 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.581103 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.581104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.581105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.581112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.582678 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.583036 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.583037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.583037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.583098 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.583271 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.583272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.583273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.583394 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.583459 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.583574 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.583651 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 23:11:04.584110 6523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.126@KRBTEST.COM
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.124@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:04.588840 6373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } attempt: 2
I20260928 23:11:04.590368 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.582180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.15.212.126:37187)
0928 23:11:04.582241 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.582243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.582252 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:04.582297 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.582298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.582308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.582383 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.582875 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.583495 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.584547 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.584813 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.584878 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.587324 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.587325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.587325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.587331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.588872 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.589297 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.589299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.589299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.589357 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.589830 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.589831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.589831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.589939 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.590140 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.590177 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.590248 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 23:11:04.591234 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.584340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54098)
0928 23:11:04.584428 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.584451 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.584501 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.584619 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.584619 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.584722 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.584748 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.584749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.585344 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.585345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.586182 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.586185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.586688 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.586689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.586728 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.587343 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.587347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.588503 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.590352 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.590353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.590353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.590528 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.590712 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.590713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.590713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.590769 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.591038 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.591039 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.591070 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.591121 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.591182 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 23:11:04.591333 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.584391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.15.212.124:33987)
0928 23:11:04.584429 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.584430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.584487 (+ 57us) server_negotiation.cc:407] Connection header received
0928 23:11:04.584548 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.584548 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.584558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.584615 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.584809 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.585287 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.586314 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.586610 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.586684 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.588586 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.588588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.588588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.588595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.590262 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.590585 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.590586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.590586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.590629 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.590823 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.590825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.590825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.590959 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.591052 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.591151 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.591246 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260928 23:11:04.593236 6523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: Bootstrap starting.
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.126@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:04.593987 6523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:04.594411 6523 log.cc:824] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:04.595602 6523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: No bootstrap required, opened a new log
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.212.125@KRBTEST.COM
I20260928 23:11:04.597277 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.589052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54106)
0928 23:11:04.589100 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.589111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.589171 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.589522 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.589524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.589682 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.589721 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.589723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.590719 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.590720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.591846 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.591848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.592305 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.592306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.592366 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.592749 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.592754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.594169 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.596073 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.596074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.596075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.596308 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.596561 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.596562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.596562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.596630 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.596940 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.596941 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.596984 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.597068 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.597160 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 23:11:04.597308 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.589246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.15.212.124:33987)
0928 23:11:04.589384 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.589385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.589391 (+ 6us) server_negotiation.cc:407] Connection header received
0928 23:11:04.589415 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.589416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.589424 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.589494 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.589803 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.590604 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.591940 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.592216 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.593230 (+ 1014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.594255 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.594256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.594256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.594280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.595986 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.596389 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.596391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.596391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.596480 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.596702 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.596703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.596703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.596858 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.596930 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.597106 (+ 176us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.597183 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260928 23:11:04.597643 6523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.597817 6523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d7a5d0a5e514575addfdf8ac122b4f9, State: Initialized, Role: FOLLOWER
I20260928 23:11:04.597903 6523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.598199 6523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:04.598153 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.598372 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.599476 6373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0: Bootstrap starting.
W20260928 23:11:04.600088 6541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:11:04.600147 6541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:11:04.600169 6373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:04.600421 6373 log.cc:824] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:04.600632 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.592506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.15.212.125:41971)
0928 23:11:04.592550 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.592552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.592560 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:11:04.592581 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.592582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.592591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.592657 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.593215 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.593683 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.594599 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.594896 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.594952 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.597646 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.597647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.597647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.597654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.599377 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.599829 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.599830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.599830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.599889 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.600099 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.600100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.600100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.600230 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.600400 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.600459 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.600530 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 23:11:04.601197 6373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0: No bootstrap required, opened a new log
I20260928 23:11:04.601928 16209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 23:11:04.602792 6347 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44866
I20260928 23:11:04.603133 6373 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.603267 6373 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7064e7918ba4a79bb231d1e5a0ab3d0, State: Initialized, Role: FOLLOWER
I20260928 23:11:04.603351 6373 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.600811 6455 master_runner.cc:186] Error getting master registration for 127.15.212.126:37187: OK, instance_id { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" instance_seqno: 1790637064247774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:11:04.603724 6455 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:11:04.603662 6373 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:04.603690 6347 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 23:11:04.604591 6347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:04.605409 6347 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.605650 6347 leader_election.cc:288] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [CANDIDATE]: Term 1 election: Requested vote from peers edad5c2b2ac940929c4e07bbb19ba53a (127.15.212.125:41971), 7d7a5d0a5e514575addfdf8ac122b4f9 (127.15.212.124:33987)
I20260928 23:11:04.603642 6544 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.605901 6544 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.606076 6497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9"
I20260928 23:11:04.606178 6497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:04.606971 6497 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7064e7918ba4a79bb231d1e5a0ab3d0 in term 1.
I20260928 23:11:04.607278 6322 leader_election.cc:302] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d7a5d0a5e514575addfdf8ac122b4f9, d7064e7918ba4a79bb231d1e5a0ab3d0; no voters:
I20260928 23:11:04.607523 6544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 23:11:04.607658 6544 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [term 1 LEADER]: Becoming Leader. State: Replica: d7064e7918ba4a79bb231d1e5a0ab3d0, State: Running, Role: LEADER
I20260928 23:11:04.607735 6544 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.608125 6544 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7064e7918ba4a79bb231d1e5a0ab3d0. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.608191 6544 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:04.609336 6305 master_runner.cc:186] Error getting master registration for 127.15.212.125:41971: OK, instance_id { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" instance_seqno: 1790637064418128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 23:11:04.612249 6305 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 23:11:04.613250 6565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 23:11:04.614660 6565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 23:11:04.615442 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.607206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.15.212.125:41971)
0928 23:11:04.607242 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.607244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.607831 (+ 587us) server_negotiation.cc:407] Connection header received
0928 23:11:04.607897 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.607898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.607907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.607951 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.608718 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.609174 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.610188 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.610968 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.611014 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.612534 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.612536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.612536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.612543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.614191 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.614581 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.614582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.614583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.614642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.614855 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.614856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.614856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.614983 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.615053 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.615267 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.615342 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 23:11:04.618283 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.607195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44870 (local address 127.15.212.126:37187)
0928 23:11:04.607395 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.607398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.607407 (+ 9us) server_negotiation.cc:407] Connection header received
0928 23:11:04.607425 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.607426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.607435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.607488 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.609870 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.610580 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.612554 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.612825 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.612863 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.615449 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.615450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.615451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.615457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.617043 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.617452 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.617453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.617453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.617505 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.617708 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.617709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.617709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.617828 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.617906 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.618117 (+ 211us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.618190 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260928 23:11:04.619789 6497 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [term 1 FOLLOWER]: Refusing update from remote peer d7064e7918ba4a79bb231d1e5a0ab3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:11:04.620137 6544 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 23:11:04.621454 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7064e7918ba4a79bb231d1e5a0ab3d0. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.621567 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.623755 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.607604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.15.212.124:33987)
0928 23:11:04.607643 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.607645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.616614 (+ 8969us) server_negotiation.cc:407] Connection header received
0928 23:11:04.616712 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.616713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.616725 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.616758 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.617368 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.618073 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.619014 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.619325 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.619368 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.620813 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.620815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.620815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.620822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.622510 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.622899 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.622900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.622900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.622975 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.623209 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.623210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.623210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.623343 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.623435 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.623576 (+ 141us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.623669 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:11:04.629458 6562 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.629550 6562 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: This master's current role is: LEADER
I20260928 23:11:04.629979 6567 mvcc.cc:205] Tried to move back new op lower bound from 7334449416680103936 to 7334449416633966592. Current Snapshot: MvccSnapshot[applied={T|T < 7334449416680103936}]
I20260928 23:11:04.630070 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.630144 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.630313 6568 mvcc.cc:205] Tried to move back new op lower bound from 7334449416680103936 to 7334449416633966592. Current Snapshot: MvccSnapshot[applied={T|T < 7334449416680103936}]
I20260928 23:11:04.632966 6565 catalog_manager.cc:1347] Generated new cluster ID: fa3bbd23bc454a68bae0c832e3349df3
I20260928 23:11:04.633085 6565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 23:11:04.652208 6448 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } attempt: 3
Sep 28 23:11:04 dist-test-slave-x2t8 krb5kdc[6292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790637064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.212.125@KRBTEST.COM for kudu/127.15.212.124@KRBTEST.COM
I20260928 23:11:04.657673 6565 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 23:11:04.659901 6565 catalog_manager.cc:1504] Loading token signing keys...
I20260928 23:11:04.660457 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.652511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.212.124:33987 (local address 127.0.0.1:54138)
0928 23:11:04.652556 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 23:11:04.652566 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 23:11:04.652602 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 23:11:04.652762 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 23:11:04.652763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 23:11:04.652948 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0928 23:11:04.652992 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.652994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.653874 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.653875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.654689 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 23:11:04.654691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.655226 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.655227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 23:11:04.655285 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.655760 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 23:11:04.655766 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 23:11:04.657460 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 23:11:04.659371 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.659372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.659373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.659575 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.659765 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 23:11:04.659766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 23:11:04.659766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 23:11:04.659819 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 23:11:04.660139 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 23:11:04.660140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 23:11:04.660183 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 23:11:04.660245 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 23:11:04.660335 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260928 23:11:04.660931 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:04.652571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.15.212.124:33987)
0928 23:11:04.652611 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.652614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.652624 (+ 10us) server_negotiation.cc:407] Connection header received
0928 23:11:04.652662 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.652663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.652671 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.652716 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.653070 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.653788 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.654780 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:04.655108 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:04.655173 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:04.657558 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 23:11:04.657559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 23:11:04.657560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 23:11:04.657566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 23:11:04.659280 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.659651 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.659652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.659652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.659703 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 23:11:04.659881 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 23:11:04.659882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 23:11:04.659882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 23:11:04.659996 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 23:11:04.660064 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:04.660772 (+ 708us) server_negotiation.cc:299] Negotiation successful
0928 23:11:04.660847 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 23:11:04.662642 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: Bootstrap starting.
I20260928 23:11:04.663415 6448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: Neither blocks nor log segments found. Creating new log.
I20260928 23:11:04.663715 6448 log.cc:824] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: Log is configured to *not* fsync() on all Append() calls
I20260928 23:11:04.669756 6565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0: Generated new TSK 0
I20260928 23:11:04.670008 6565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 23:11:04.673074 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: No bootstrap required, opened a new log
I20260928 23:11:04.675484 6448 raft_consensus.cc:348] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.675787 6448 raft_consensus.cc:729] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edad5c2b2ac940929c4e07bbb19ba53a, State: Initialized, Role: FOLLOWER
I20260928 23:11:04.675976 6448 consensus_queue.cc:260] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } }
I20260928 23:11:04.676637 6421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [term 0 FOLLOWER]: Advancing to term 1
I20260928 23:11:04.676599 6422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "edad5c2b2ac940929c4e07bbb19ba53a"
I20260928 23:11:04.676927 6448 sys_catalog.cc:564] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 23:11:04.677572 6421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [term 1 FOLLOWER]: Refusing update from remote peer d7064e7918ba4a79bb231d1e5a0ab3d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 23:11:04.676471 6573 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.677776 6573 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.678386 6422 raft_consensus.cc:2454] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7064e7918ba4a79bb231d1e5a0ab3d0 in term 1.
I20260928 23:11:04.679824 6544 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 23:11:04.685392 6584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 23:11:04.685474 6584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 23:11:04.686067 6589 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7064e7918ba4a79bb231d1e5a0ab3d0. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.686167 6589 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.686194 6573 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:04.686268 6573 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 23:11:04.686944 6587 mvcc.cc:205] Tried to move back new op lower bound from 7334449416883388416 to 7334449416633966592. Current Snapshot: MvccSnapshot[applied={T|T < 7334449416680103936}]
W20260928 23:11:04.704389 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.702788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.15.212.125:41971)
0928 23:11:04.702830 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.702832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.703654 (+ 822us) server_negotiation.cc:407] Connection header received
0928 23:11:04.703760 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.703761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.703773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.703804 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.704311 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50146: BlockingRecv error: recv got EOF from 127.0.0.1:50146 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 23:11:04.705222 6528 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.703155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.15.212.124:33987)
0928 23:11:04.703195 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.703198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.704656 (+ 1458us) server_negotiation.cc:407] Connection header received
0928 23:11:04.704748 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.704749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.704759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.704786 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.705153 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54154: BlockingRecv error: recv got EOF from 127.0.0.1:54154 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 23:11:04.705901 6375 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.703504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.15.212.126:37187)
0928 23:11:04.703541 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.703543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.705440 (+ 1897us) server_negotiation.cc:407] Connection header received
0928 23:11:04.705535 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.705536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.705546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.705574 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.705844 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44884: BlockingRecv error: recv got EOF from 127.0.0.1:44884 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 23:11:04.707767 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.707083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.15.212.125:41971)
0928 23:11:04.707114 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.707116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.707293 (+ 177us) server_negotiation.cc:407] Connection header received
0928 23:11:04.707391 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.707392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.707407 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.707434 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.707727 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50154: BlockingRecv error: recv got EOF from 127.0.0.1:50154 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 23:11:04.710670 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.709925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.15.212.125:41971)
0928 23:11:04.709956 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.709957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.710194 (+ 237us) server_negotiation.cc:407] Connection header received
0928 23:11:04.710288 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.710289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.710298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.710326 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.710631 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50158: BlockingRecv error: recv got EOF from 127.0.0.1:50158 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 23:11:04.714463 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.713728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.15.212.125:41971)
0928 23:11:04.713760 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.713761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.713980 (+ 219us) server_negotiation.cc:407] Connection header received
0928 23:11:04.714092 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.714093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.714103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.714131 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.714424 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50174: BlockingRecv error: recv got EOF from 127.0.0.1:50174 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 23:11:04.720333 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.719573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.15.212.125:41971)
0928 23:11:04.719607 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.719609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.719826 (+ 217us) server_negotiation.cc:407] Connection header received
0928 23:11:04.719924 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.719925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.719935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.719965 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.720293 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50184: BlockingRecv error: recv got EOF from 127.0.0.1:50184 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 23:11:04.730338 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.729536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.15.212.125:41971)
0928 23:11:04.729572 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.729573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.729804 (+ 231us) server_negotiation.cc:407] Connection header received
0928 23:11:04.729902 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.729903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.729914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.729942 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.730290 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50192: BlockingRecv error: recv got EOF from 127.0.0.1:50192 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 23:11:04.748656 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.747685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50202 (local address 127.15.212.125:41971)
0928 23:11:04.747726 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.747728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.747989 (+ 261us) server_negotiation.cc:407] Connection header received
0928 23:11:04.748109 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.748110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.748121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.748153 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.748594 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50202: BlockingRecv error: recv got EOF from 127.0.0.1:50202 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 23:11:04.782807 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.781997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50206 (local address 127.15.212.125:41971)
0928 23:11:04.782054 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.782056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.782218 (+ 162us) server_negotiation.cc:407] Connection header received
0928 23:11:04.782297 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.782298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.782311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.782383 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.782735 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50206: BlockingRecv error: recv got EOF from 127.0.0.1:50206 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 23:11:04.848877 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.848075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.15.212.125:41971)
0928 23:11:04.848147 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.848149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.848271 (+ 122us) server_negotiation.cc:407] Connection header received
0928 23:11:04.848351 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.848352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.848364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.848436 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.848812 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50212: BlockingRecv error: recv got EOF from 127.0.0.1:50212 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260928 23:11:04.978670 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:04.978020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.15.212.125:41971)
0928 23:11:04.978089 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:04.978090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:04.978163 (+ 73us) server_negotiation.cc:407] Connection header received
0928 23:11:04.978213 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:04.978214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:04.978224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:04.978301 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:04.978621 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50214: BlockingRecv error: recv got EOF from 127.0.0.1:50214 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 23:11:05.079830 6544 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7064e7918ba4a79bb231d1e5a0ab3d0" member_type: VOTER last_known_addr { host: "127.15.212.126" port: 37187 } } peers { permanent_uuid: "edad5c2b2ac940929c4e07bbb19ba53a" member_type: VOTER last_known_addr { host: "127.15.212.125" port: 41971 } } peers { permanent_uuid: "7d7a5d0a5e514575addfdf8ac122b4f9" member_type: VOTER last_known_addr { host: "127.15.212.124" port: 33987 } } }
I20260928 23:11:05.080005 6544 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7064e7918ba4a79bb231d1e5a0ab3d0 [sys.catalog]: This master's current role is: LEADER
W20260928 23:11:05.236527 6452 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 23:11:05.235898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.15.212.125:41971)
0928 23:11:05.235963 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.235965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.235976 (+ 11us) server_negotiation.cc:407] Connection header received
0928 23:11:05.236023 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.236024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.236033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.236093 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 23:11:05.236422 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50226: BlockingRecv error: recv got EOF from 127.0.0.1:50226 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 23:11:05.600589 6541 catalog_manager.cc:1259] Loaded cluster ID: fa3bbd23bc454a68bae0c832e3349df3
I20260928 23:11:05.600664 6541 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: loading cluster ID for follower catalog manager: success
I20260928 23:11:05.601418 6541 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: acquiring CA information for follower catalog manager: success
I20260928 23:11:05.601662 6541 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d7a5d0a5e514575addfdf8ac122b4f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:11:05.685950 6584 catalog_manager.cc:1259] Loaded cluster ID: fa3bbd23bc454a68bae0c832e3349df3
I20260928 23:11:05.686018 6584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: loading cluster ID for follower catalog manager: success
I20260928 23:11:05.686898 6584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: acquiring CA information for follower catalog manager: success
I20260928 23:11:05.687219 6584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P edad5c2b2ac940929c4e07bbb19ba53a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 23:11:05.752045 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:05.749623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.15.212.125:41971)
0928 23:11:05.749770 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.749772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.749786 (+ 14us) server_negotiation.cc:407] Connection header received
0928 23:11:05.749812 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.749813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.749860 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.749944 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:05.750073 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.750574 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.751280 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.751496 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.751550 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:05.751705 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:05.751812 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:05.751878 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:05.751911 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 23:11:05.751928 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260928 23:11:05.755916 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:05.753772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.15.212.124:33987)
0928 23:11:05.753880 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.753882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.753890 (+ 8us) server_negotiation.cc:407] Connection header received
0928 23:11:05.753936 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.753936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.753972 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.754043 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:05.754124 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.754592 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.755245 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.755433 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.755481 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:05.755598 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:05.755691 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:05.755746 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:05.755791 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 23:11:05.755806 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260928 23:11:05.760269 6663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:05.757681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.15.212.124:33987)
0928 23:11:05.757726 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.757728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.757858 (+ 130us) server_negotiation.cc:407] Connection header received
0928 23:11:05.757933 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.757934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.757944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.757999 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:05.758115 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.758610 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.759509 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.759755 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.759830 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:05.760000 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:05.760040 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:05.760071 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:05.760179 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 23:11:05.760203 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 23:11:05.760306 6657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:05.757483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.15.212.125:41971)
0928 23:11:05.757546 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.757548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.757725 (+ 177us) server_negotiation.cc:407] Connection header received
0928 23:11:05.757798 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.757799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.757809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.757859 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:05.758033 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.758773 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.759597 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.759884 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.759950 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:05.760069 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:05.760098 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:05.760145 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:05.760220 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 23:11:05.760237 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 23:11:05.760946 6670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 23:11:05.757572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.15.212.126:37187)
0928 23:11:05.757685 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 23:11:05.757687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 23:11:05.757802 (+ 115us) server_negotiation.cc:407] Connection header received
0928 23:11:05.757902 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 23:11:05.757904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 23:11:05.757955 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 23:11:05.758026 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 23:11:05.758245 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.758990 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.759994 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 23:11:05.760310 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 23:11:05.760371 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 23:11:05.760613 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 23:11:05.760712 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 23:11:05.760790 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 23:11:05.760842 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 23:11:05.760864 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260928 23:11:05.768209 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 6305
I20260928 23:11:05.773334 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 6380
I20260928 23:11:05.778007 16209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR8rxt/build/release/bin/kudu with pid 6455
2026-09-28T23:11:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2775 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5563 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267023 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST